16

Edge Detection in Color Images

Edge information is essential in many image analysis and computer

vision applications and thus the ability to locate and characterize
edges robustly and accurately is an important task. Basic techniques
for edge detection in grayscale images are discussed in Chapter 6.
Color images contain richer information than grayscale images and
it appears natural to assume that edge detection methods based on
color should outperform their monochromatic counterparts. For ex-
ample, locating an edge between two image regions of different hue
but similar brightness is difficult with an edge detector that only
looks for changes in image intensity. In this chapter, we first look at
the use of “ordinary” (i.e., monochromatic) edge detectors for color
images and then discuss dedicated detectors that are specifically de-
signed for color images.
Although the problem of color edge detection has been pursued for
a long time (see [140,266] for a good overview), most image processing
texts do not treat this subject in much detail. One reason could be
that, in practice, edge detection in color images is often accomplished
by using “monochromatic” techniques on the intensity channel or
the individual color components. We discuss these simple methods—
which nevertheless give satisfactory results in many situations—in
Sec. 16.1.
Unfortunately, monochromatic techniques do not extend natu-
rally to color images and other “multi-channel” data, since edge in-
formation in the different color channels may be ambiguous or even
contradictory. For example, multiple edges running in different di-
rections may coincide at a given image location, edge gradients may
cancel out, or edges in different channels may be slightly displaced.
In Sec. 16.2, we describe how local gradients can be calculated for
edge detection by treating the color image as a 2D vector field. In
Sec. 16.3, we show how the popular Canny edge detector, originally
designed for monochromatic images, can be adapted for color images,
and Sec. 16.4 goes on to look at other color edge operators. Imple-
mentations of the discussed algorithms are described in Sec. 16.5,
with complete source code available on the book’s website.
391
© Springer-Verlag London 2016
W. Burger, M.J. Burge, Digital Image Processing, Texts in Computer Science,
DOI 10.1007/978-1-4471-6684-9_16
16 Edge Detection in 16.1 Monochromatic Techniques
Color Images
Linear filters are the basis of most edge enhancement and edge de-
tection operators for scalar-valued grayscale images, particularly the
gradient filters described in Chapter 15, Sec. 6.3. Again, it is quite
common to apply these scalar filters separately to the individual color
channels of RGB images. A popular example is the Sobel operator
with the filter kernels
⎡ ⎤ ⎡ ⎤
−1 0 1 −1 −2 −1
1 1
HxS = · ⎣ −2 0 2 ⎦ and HyS = · ⎣ 0 0 0 ⎦ (16.1)
8 8
−1 0 1 1 2 1

for the x- and y-direction, respectively. Applied to a grayscale image

I, with Ix = I ∗ HxS and Iy = I ∗ HyS , these filters give a reasonably
good estimate of the local gradient vector,
! "
Ix (u)
∇I(u) = , (16.2)
Iy (u)

at position u = (u, v). The local edge strength of the grayscale image
is then taken as
%
Egray (u) = ∇I(u) = Ix2 (u) + Iy2 (u), (16.3)

and the corresponding edge orientation is calculated as

 I (u) 
y
Φ(u) = ∠∇I(u) = tan−1 . (16.4)
Ix (u)

The angle Φ(u) gives the direction of maximum intensity change on

the 2D image surface at position (u), which is the normal to the edge
tangent.
Analogously, to apply this technique to a color image I = (IR ,
IG , IB ), each color plane is first filtered individually with the two
gradient kernels given in Eqn. (16.1), resulting in
! " ! "
I I ∗ HxS
∇IR = R,x = R ,
IR,y IR ∗ HyS
! " ! "
I I ∗ HxS
∇IG = G,x = G , (16.5)
IG,y IG ∗ HyS
! " ! "
I I ∗ HxS
∇IB = B,x = B .
IB,y IB ∗ HyS

The local edge strength is calculated separately for each color channel
which yields a vector
⎛ ⎞ ⎛ ⎞
ER (u) ∇IR (u)
E(u) = ⎝EG (u)⎠ = ⎝∇IG (u)⎠ (16.6)
EB (u) ∇IB (u)
⎛ ⎞
[IR,x
2
(u) + IR,y
2
(u)]1/2
= ⎝[IG,x (16.7)
⎜ 2 ⎟
(u) + IG,y
2
(u)]1/2 ⎠
[IB,x (u) + IB,y (u)]
2 2 1/2
392
for each image position u. These vectors could be combined into 16.1 Monochromatic
a new color image E = (ER , EG , EB ), although such a “color edge Techniques
image” has no particularly useful interpretation.1 Finally, a scalar
quantity of combined edge strength (C) over all color planes can be
obtained, for example, by calculating the Euclidean (L2 ) norm of
E as
2 2
1/2
C2 (u) = E(u)2 = ER (u) + EG (u) + EB2 (u)
2 2 2 2 2 2
1/2
= IR,x + IR,y + IG,x + IG,y + IB,x + IB,y (16.8)

(coordinates (u) are omitted in the second line) or, using the L1
norm,

C1 (u) = E(u)1 = |ER (u)| + |EG (u)| + |EB (u)| . (16.9)

Another alternative for calculating a combined edge strength is to

take the maximum magnitude of the RGB gradients (i.e., the L∞
norm),

C∞ (u) = E(u)∞ = max (|ER (u)| , |EG (u)| , |EB (u)|) . (16.10)

An example using the test image from Chapter 15 is given in Fig.

16.1. It shows the edge magnitude of the corresponding grayscale
image and the combined color edge magnitude calculated with the
different norms defined in Eqns. (16.8)–(16.10).2
As far as edge orientation is concerned, there is no simple ex-
tension of the grayscale case. While edge orientation can easily be
calculated for each individual color component (using Eqn. (16.4)),
the gradients, three color channels are generally different (or even
contradictory) and there is no obvious way of combining them.
A simple ad hoc approach is to choose, at each image position
u, the gradient direction from the color channel of maximum edge
strength, that is,

m,y (u)
I 
Φcol (u) = tan−1 , (16.11)
Im,x (u)

with m = argmax Ek (u).

k=R,G,B
This simple (monochromatic) method for calculating edge strength
and orientation in color images is summarized in Alg. 16.1 (see Sec.
16.5 for the corresponding Java implementation). Two sample results
are shown in Fig. 16.2. For comparison, these figures also show the
edge maps obtained by first converting the color image to a grayscale
1
Such images are nevertheless produced by the “Find Edges” command in
ImageJ and the filter of the same name in Photoshop (showing inverted
components).
2
In this case, the grayscale image in (c) was calculated with the direct
conversion method (see Chapter 14, Eqn. (14.39)) from nonlinear sRGB
components. With linear grayscale conversion (Ch. 14, Eqn. (14.37)),
the desaturated bar at the center would exhibit no grayscale edges along
its borders, since the luminance is the same inside and outside.
393
16 Edge Detection in
Color Images

Fig. 16.1
Color edge enhancement (a) (b)
with monochromatic meth-
ods. Original color image (a)
and corresponding grayscale
image (b); edge magnitude
from the grayscale image (c).
Color edge magnitude calcu-
lated with different norms:
L1 (d), L2 (e), and L∞ (f). sRGB Y′
The images in (c–f) are in-
verted for better viewing.

(c) (d)

Egray C1

(e) (f)

C2 C∞

image and then applying the Sobel operator3 (Fig. 16.2(b)). The
edge magnitude in all examples is normalized; it is shown inverted
and contrast-enhanced to increase the visibility of low-contrast edges.
As expected and apparent from the examples, even simple monochro-
matic techniques applied to color images perform better than edge
detection on the corresponding grayscale images. In particular, edges
between color regions of similar brightness are not detectable in this
way, so using color information for edge detection is generally more
powerful than relying on intensity alone. Among the simple color
techniques, the maximum channel edge strength C∞ (Eqn. (16.10))
seems to give the most consistent results with the fewest edges getting
lost.
However, none of the monochromatic detection techniques can
be expected to work reliably under these circumstances. While the
threshold for binarizing the edge magnitude could be tuned manu-
ally to give more pleasing results on specific images, it is difficult
in practice to achieve consistently good results over a wide range of
images. Methods for determining the optimal edge threshold dynam-

3
See Chapter 6, Sec. 6.3.1.
394
 1: MonochromaticColorEdge(I) 16.2 Edges in
Input: I = (IR , IG , IB ), an RGB color image of size M ×N . Re- Vector-Valued Images
turns a pair of maps (E2 , Φ) for edge magnitude and orientation.
- −1 0 1
. Alg. 16.1
2: HxS ← 1
8
· −2 0 2 Monochromatic color edge
−1 0 1 operator. A pair of Sobel-
- −1 −2 −1 . type filter kernels (HxS , HyS )
3: HyS ← 1
8
· 0 0 0 ⊲ x/y gradient kernels is used to estimate the local
1 2 1 x/y gradients of each compo-
nent of the RGB input image
4: (M, N ) ← Size(I) I. Color edge magnitude is
5: Create maps E, Φ : M ×N → R ⊲ edge magnitude/orientation calculated as the L2 norm of
the color gradient vector (see
6: IR,x ← IR ∗HxS , IR,y ← IR ∗HyS ⊲ apply gradient filters Eqn. (16.8)). The procedure
7: IG,x ← IG ∗HxS , IG,y ← IG ∗HyS returns a pair of maps, hold-
ing the edge magnitude E2
8: IB,x ← IB ∗HxS , IB,y ← IB ∗HyS and the edge orientation Φ,
respectively.
9: for all image coordinates u ∈ M ×N do
10: (rx , gx , bx ) ← (IR,x (u), IG,x (u), IB,x (u))
11: (ry , gy , by ) ← (IR,y (u), IG,y (u), IB,y (u))
12: e2R ← rx2 + ry2
13: e2G ← gx2 + gy2
14: e2B ← b2x + b2y
15: e2max ← e2R ⊲ find maximum gradient channel
16: cx ← rx , cy ← ry
17: if e2G > e2max then
18: e2max ← e2G , cx ← gx , cy ← gy
19: if e2B > e2max then
20: e2max ← e2B , cx ← bx , cy ← by

21: E(u) ← + e2R
+ e2G e2B ⊲ edge magnitude (L2 norm)
22: Φ(u) ← ArcTan(cx , cy ) ⊲ edge orientation
23: return (E, Φ).

ically, that is, depending on the image content, have been proposed,
typically based on the statistical variability of the color gradients.
Additional details can be found in [84, 171, 192].

16.2 Edges in Vector-Valued Images

In the “monochromatic” scheme described in Sec. 16.1, the edge mag-
nitude in each color channel is calculated separately and thus no use
is made of the potential coupling between color channels. Only in a
subsequent step are the individual edge responses in the color chan-
nels combined, albeit in an ad hoc fashion. In other words, the color
data are not treated as vectors, but merely as separate and unrelated
scalar values.
To obtain better insight into this problem it is helpful to treat the
color image as a vector field, a standard construct in vector calculus
[32, 223].4 A three-channel RGB color image I(u) = (IR (u), IG (u),
IB (u)) can be modeled as a discrete 2D vector field, that is, a function
whose coordinates u = (u, v) are 2D and whose values are 3D vectors.

4
See Sec. C.2 in the Appendix for some general properties of vector fields.
395
 16 Edge Detection in
Color Images

Fig. 16.2 (a)

Example of color edge en-
hancement with monochro-
matic techniques (balloons im-
age). Original color image and
corresponding grayscale image
(a), edge magnitude obtained
from the grayscale image (b), Original image I
color edge magnitude calcu-
lated with the L2 norm (c),
and the L∞ norm (d). Differ-
ences between the grayscale
edge detector (b) and the
color-based detector (c–e) are
particularly visible inside the (b)
right balloon and at the lower
borders of the tangerines.

Gray edge (Egray )

(c)

Color edge (E1 )

(d)

Color edge (E2 )

(e)

Color edge (C∞ )

Similarly, a grayscale image can be described as a discrete scalar field,

since its pixel values are only 1D.

396
16.2.1 Multi-Dimensional Gradients 16.2 Edges in
Vector-Valued Images
As noted in the previous section, the gradient of a scalar image I at
a specific position u is defined as
# $
∂I
∂x (u)
∇I(u) = ∂I , (16.12)
∂y (u)

that is, the vector of the partial derivatives of the function I in the
x- and y-direction, respectively.5 Obviously, the gradient of a scalar
image is a 2D vector field.
In the case of a color image I = (IR , IG , IB ), we can treat the three
color channels as separate scalar images and obtain their gradients
analogously as
# ∂I $ # ∂I $ # ∂I $
∂x (u) ∂x (u) ∂x (u)
R G B

∇IR (u) = ∂I , ∇IG (u) = ∂I , ∇IB (u) = ∂I ,

∂y (u) ∂y (u) ∂y (u)
R G B

(16.13)
which is equivalent to what we did in Eqn. (16.5). Before we can take
the next steps, we need to introduce a standard tool for the analysis
of vector fields.

16.2.2 The Jacobian Matrix

The Jacobian matrix6 JI (u) combines all first partial derivatives of a
vector field I at a given position u, its row vectors being the gradients
of the scalar component functions. In particular, for an RGB color
image I, the Jacobian matrix is defined as
⎛ ⎞ ⎛ ∂I ∂IR ⎞
(∇IR )⊺ (u) ∂x (u) ∂y (u)
R

⎜ ⊺
⎟ ⎜ ∂I ∂IG
⎟  
⎝(∇IG ) (u)⎠ = ⎝ ∂x (u) ∂y (u)⎠ = I x (u) I y (u) ,
JI (u) = ⎜ ⎟ ⎜ G ⎟
(∇IB )⊺ (u) ∂IB ∂IB
∂x (u) ∂y (u) (16.14)
with ∇IR , ∇IG , ∇IB as defined in Eqn. (16.13). We see that the 2D
gradient vectors (∇IR )⊺ , (∇IG )⊺ , (∇IB )⊺ constitute the rows of the
resulting 3 × 2 matrix JI . The two 3D column vectors of this matrix,
⎛ ∂I ⎞ ⎛ ∂I ⎞
∂x (u) ∂y (u)
R R

∂I ⎜ ∂I ⎟ ∂I ⎜ ∂I ⎟
I x (u) = (u) = ⎜ ∂x
G
(u)⎟ , I y (u) = (u) = ⎜ G
∂y (u)⎟
⎠,
∂x ⎝ ⎠ ∂y ⎝
∂IB ∂I
∂x (u) ∂y (u)
B

(16.15)
are the partial derivatives of the color components along the x- and y-
axes, respectively. At a given position u, the total amount of change
over all three color channels in the horizontal direction can be quanti-
fied by the norm of the corresponding column vector I x (u). Anal-
ogously, I y (u) gives the total amount of change over all three color
channels along the vertical axis.
5
Of course, images are discrete functions and the partial derivatives are
estimated from finite differences (see Sec. C.3.1 in the Appendix).
6
See also Sec. C.2.1 in the Appendix.
397
16 Edge Detection in 16.2.3 Squared Local Contrast
Color Images
Now that we can quantify the change along the horizontal and vertical
axes at any position u, the next task is to find out the direction of
the maximum change to find the angle of the edge normal, which
we then use to derive the local edge strength. How can we calculate
the gradient in some direction θ other than horizontal and vertical?
For this purpose, we use the product of the unit vector oriented at
angle θ,
! "
cos(θ)
eθ = , (16.16)
sin(θ)

and the Jacobian matrix JI (Eqn. (16.14)) in the form

 ! "
cos(θ)

(gradθ I)(u) = JI (u) · eθ = I x (u) I y (u) ·
sin(θ) (16.17)
= I x (u) · cos(θ) + I y (u) · sin(θ).
The resulting 3D vector (gradθ I)(u) is called the directional gradi-
ent 7 of the color image I in the direction θ at position u. By taking
the squared norm of this vector,
2
Sθ (I, u) = (gradθ I)(u)2 (16.18)
5 52
= 5I x (u)·cos(θ) + I y (u)·sin(θ)52
= I 2x (u) · cos2 (θ) + 2·I x (u)·I y (u)·cos(θ)·sin(θ) + I 2y (u)·sin2 (θ),
we obtain what is called the squared local contrast of the vector-
valued image I at position u in direction θ.8 For an RGB image I =
(IR , IG , IB ), the squared local contrast in Eqn. (16.18) is, explicitly
written,
⎛ ⎞ ⎛ ⎞
5 IR,x (u) IR,y (u) 52
Sθ (I, u) = 5⎝IG,x (u)⎠ · cos(θ) + ⎝IG,y (u)⎠ · sin(θ)52 (16.19)
IB,x (u) IB,y (u)
2 2 2

= IR,x (u) + IG,x (u) + IB,x (u) · cos2 (θ)
2 2 2

+ IR,y (u) + IG,y (u) + IB,y (u) · sin2 (θ) (16.20)
+ 2 · cos(θ) · sin(θ) · 
IR,x (u)·IR,y (u)+IG,x (u)·IG,y (u)+IB,x (u)·IB,y (u) .
Note that, in the case that I is a scalar image, the squared local
contrast reduces to
52 5 I (u) ⊺ cos(θ) 52
! " ! "
5
Sθ (I, u) = 5(gradθ I)(u)5 = 5 x · 5 (16.21)
Iy (u) sin(θ) 2
2
= Ix (u) · cos(θ) + Iy (u) · sin(θ) . (16.22)
We will return to this result again later in Sec. 16.2.6. In 
the follow-
ing, we use the root of the squared local contrast, that is, Sθ (I, u),
under the term local contrast.
7
See also Sec. C.2.2 in the Appendix (Eqn. (C.18)).
8
Note that I 2x = I x ·I x , I 2y = I y ·I y and I x ·I y in Eqn. (16.18) are dot
products and thus the results are scalar values.
398
 Grayscale image I RGB color image I = (IR , IG , IB )
16.2 Edges in
Vector-Valued Images

Fig. 16.3
∇IR Local image gradients and lo-
cal contrast. In case of a scalar
∇I (grayscale) image I (a), the lo-
∇IB cal gradient ∇I defines a single
∇IG plane that is tangential to the
image function I at position
u = (u, v). In case of an RGB
color image I = (IR , IG , IB )
(b), the local gradients ∇IR ,
∇IG , ∇IB for each color
channel define three tangent
(a) (b) planes. The vertical axes in
graphs (c, d) show the corre-
sponding
 local contrast values
Sθ (I, u) (see Eqns. (16.18)
and (16.19)) for all possible
directions θ = 0, . . . , 2π.

 
Sθ (u) Sθ (u)

(c) (d)

Figure 16.3 illustrates the meaning of the squared local contrast

in relation to the local image gradients. At a given image position u,
the local gradient ∇I(u) in a grayscale image (Fig. 16.3(a)) defines a
single plane that is tangential to the image function I at position u.
In case of a color image (Fig. 16.3(b)), each color channel defines an
individual tangent plane. In Fig. 16.3(c, d) the local contrast values
are shown as the height of cylindrical surfaces for all directions θ. For
a grayscale image (Fig. 16.3(c)), the local contrast changes linearly
with the orientation θ, while the relation is quadratic for a color image
(Fig. 16.3(d)). To calculate the strength and orientation of edges we
need to determine the direction of the maximum local contrast, which
is described in the following.

16.2.4 Color Edge Magnitude

The directions that maximize Sθ (I, u) in Eqn. (16.18) can be found
analytically as the roots of the first partial derivative of S with respect
to the angle θ, as originally suggested by Di Zenzo [63], and the
resulting quantity is called maximum local contrast. As shown in [59],
the maximum local contrast can also be found from the Jacobian
matrix JI (Eqn. (16.14)) as the largest eigenvalue of the (symmetric)
2 × 2 matrix
# ⊺ $
⊺ I x (u)  
M(u) = JI (u)· JI (u) = ⊺ · I x (u) I y (u) (16.23)
I y (u)
! " ! "
I 2x (u) I x (u)·I y (u) A(u) C(u)
= = , (16.24)
I y (u)·I x (u) I 2y (u) C(u) B(u)
399
16 Edge Detection in with the elements
Color Images
A(u) = I 2x (u) = I x (u)·I x (u),
B(u) = I 2y (u) = I y (u)·I y (u), (16.25)
C(u) = I x (u)·I y (u) = I y (u)·I x (u).

The matrix M(u) could be considered as the color equivalent to the

local structure matrix used for corner detection on grayscale images
in Chapter 7, Sec. 7.2.1. The two eigenvalues λ1 , λ2 of M can be
found in closed form as9
  
λ1 (u) = A + B + (A − B)2 + 4 · C 2 /2,
   (16.26)
λ2 (u) = A + B − (A − B)2 + 4 · C 2 /2.

Since M is symmetric, the expression under the square root in Eqn.

(16.26) is positive and thus all eigenvalues are real. In addition, A, B
are both positive and therefore λ1 is always the larger of the two
eigenvalues. It is equivalent to the maximum squared local contrast
(Eqn. (16.18)), that is,

λ1 (u) ≡ max Sθ (I, u), (16.27)

0≤θ<2π
√
and thus λ1 can be used directly to quantify the local edge strength.
The eigenvector associated with λ1 (u) is
!  "
A − B + (A − B)2 + 4 · C 2
q 1 (u) = , (16.28)
2·C

or, equivalently, any multiple of q 1 .10 Thus the rate of change along
the vector q 1 is the same as in the opposite direction −q1 , and it
follows that the local contrast Sθ (I, u) at orientation θ is the same
at orientation θ + kπ (for any k ∈ Z).11 As usual, the unit vec-
tor corresponding to q 1 is obtained by scaling q 1 by its magnitude,
that is,
1
q̂ 1 = ·q . (16.29)
q 1  1
An alternative method, proposed in [60], is to calculate the unit
eigenvector q̂ 1 = (x̂1 , ŷ1 )⊺ in the form
% % ⊺
q̂ 1 = 1+α
2 , sgn(C)· 1−α ,
2
(16.30)

with α = (A − B)/ (A − B)2 + 4 C 2 , directly from the matrix ele-
ments A, B, C defined in Eqn. (16.25).
While q 1 (the eigenvector associated with the greater eigenvalue
of M) points in the direction of maximum change, the second eigen-
vector q 2 (associated with λ2 ) is orthogonal to q 1 , that is, has the
same direction as the local edge tangent.
9
See Sec. B.4 in the Appendix for details.
10
The eigenvalues of a matrix are unique, but the corresponding eigenvec-
tors are not.
11
Thus the orientation of maximum change is inherently ambiguous [60].
400
16.2.5 Color Edge Orientation 16.2 Edges in
Vector-Valued Images
The local orientation of the edge (i.e., the normal to the edge tangent)
at a given position u can be obtained directly from the associated
eigenvector q 1 (u) = (qx (u), qy (u))⊺ using the relation
qx (u) 2·C
tan(θ1 (u)) = =  , (16.31)
qy (u) A − B + (A − B)2 + 4 · C 2
which can be simplified12 to
2·C
tan(2 · θ1 (u)) = . (16.32)
A−B
Unless both A = B and C = 0 (in which case the edge orientation
is undetermined) the angle of maximum local contrast or color edge
orientation can be calculated as
1  2·C  1
θ1 (u) = · tan−1 = · ArcTan(A − B, 2 · C). (16.33)
2 A−B 2
The above steps are summarized in Alg. 16.2, which is a color edge
operator based on the first derivatives of the image function (see Sec.
16.5 for the corresponding Java implementation). It is similar to the
algorithm proposed by Di Zenzo [63] but uses the eigenvalues of the
local structure matrix for calculating edge magnitude and orientation,
as suggested in [59] (see Eqn. (16.24)).
Results of the monochromatic edge operator in Alg. 16.1 and the
Di Zenzo-Cumani multi-gradient operator in Alg. 16.2 are compared
in Fig. 16.4. The synthetic test image in Fig. 16.4(a) has constant
luminance (brightness) and thus no gray-value operator should be
able to detect edges in this image. The local edge strength E(u)
produced by the two operators is very similar (Fig. 16.4(b)). The
vectors in Fig. 16.4(c–f) show the orientation of the edge tangents
that are normals to the direction of maximum color contrast, Φ(u).
The length of each tangent vector is proportional to the local edge
strength E(u).
Figure 16.5 shows two examples of applying the Di Zenzo-Cumani-
style color edge operator (Alg. 16.2) to real images. Note that the
multi-gradient edge magnitude (calculated from the eigenvalue λ1 in
Eqn. (16.27)) in Fig. 16.5(b) is virtually identical to the monochro-
matic edge magnitude Emag under the L2 norm in Fig. 16.2(d). The
larger difference to the result for the L∞ norm in Fig. 16.2(e) is shown
in Fig. 16.5(c).
Thus, considering only edge magnitude, the Di Zenzo-Cumani op-
erator has no significant advantage over the simpler, monochromatic
operator in Sec. 16.1. However, if edge orientation is important (as
in the color version of the Canny operator described in Sec. 16.3), the
Di Zenzo-Cumani technique is certainly more reliable and consistent.

16.2.6 Grayscale Gradients Revisited

As one might have guessed, the usual gradient-based calculation of
the edge orientation (see Ch. 6, Sec. 6.2) is only a special case of the
12
Using the relation tan(2θ) = [2 · tan(θ)] / [1 − tan2 (θ)].
401
 16 Edge Detection in 1: MultiGradientColorEdge(I)
Color Images Input: I = (IR , IG , IB ), an RGB color image of size M × N .
Returns a pair of maps (E, Φ) for edge magnitude and orientation.
Alg. 16.2 - −1 0 1
.
Di Zenzo/Cumani-style multi- 2: HxS := 1
8
· −2 0 2
gradient color edge operator. −1 0 1
A pair of Sobel-type filters - −1 −2 −1 .
(HxS , HyS ) is used for esti- 3: HyS := 1
8
· 0 0 0 ⊲ x/y gradient kernels
mating the local x/y gradi- 1 2 1
ents in each component of
the RGB input image I. The 4: (M, N ) ← Size(I)
procedure returns a pair of 5: Create maps E, Φ : M ×N → R ⊲ edge magnitude/orientation
maps, holding the edge mag-
nitude E(u) and the edge 6: IR,x ← IR ∗HxS , IR,y ← IR ∗HyS ⊲ apply gradient filters
orientation Φ(u), respectively.
7: IG,x ← IG ∗HxS , IG,y ← IG ∗HyS
8: IB,x ← IB ∗HxS , IB,y ← IB ∗HyS
9: for all u ∈ M ×N do
10: (rx , gx , bx ) ← (IR,x (u), IG,x (u), IB,x (u))
11: (ry , gy , by ) ← (IR,y (u), IG,y (u), IB,y (u))
12: A ← rx2 + gx2 + b2x ⊲ A = I x ·I x
13: B ← ry2 + gy2 + b2y ⊲ B = I y ·I y
14: C ← rx ·ry + gx ·gy + bx ·by ⊲ C = I x ·I y
  
15: λ1 ← A+B + (A−B)2 + 4 · C 2 / 2 ⊲ Eq. 16.26
√
16: E(u) ← λ1 ⊲ Eq. 16.27
1
17: Φ(u) ← 2
· ArcTan(A−B, 2 · C) ⊲ Eq. 16.33
18: return (E, Φ).

multi-dimensional gradient calculation described already. Given a

scalar image I, the intensity gradient vector (∇I)(u) = (Ix (u), Iy (u))⊺
defines a single plane that is tangential to the image function at po-
sition u, as illustrated in Fig. 16.3(a). With

A = Ix2 (u), B = Iy2 (u), C = Ix (u)·Iy (u) (16.34)

(analogous to Eqn. (16.25)) the squared local contrast at position u

in direction θ (as defined in Eqn. (16.18)) is
 2
Sθ (I, u) = Ix (u)·cos(θ) + Iy (u)·sin(θ) . (16.35)

From Eqn. (16.26), the eigenvalues of the local structure matrix M =

(C
A C ) at position u are (see Eqn. (16.26))
B
  
λ1,2 (u) = A + B ± (A − B)2 + 4C 2 / 2, (16.36)

but here, with Ix , Iy not being vectors but scalar values, we get C 2 =
(Ix ·Iy )2 = Ix2 ·Iy2 , such that (A−B)2 +4C 2 = (A+B)2 , and therefore
 
λ1,2 (u) = A + B ± (A + B) / 2 . (16.37)

We see that, for a scalar-valued image, the dominant eigenvalue,

2
λ1 (u) = A + B = Ix2 (u) + Iy2 (u) = ∇I(u)2 , (16.38)

is simply the squared L2 norm of the local gradient vector, while the
smaller eigenvalue λ2 is always zero. Thus, for a grayscale image, the
402
 Original image Color edge strength
16.2 Edges in
Vector-Valued Images

Fig. 16.4
Results from the monochro-
matic (Alg. 16.1) and the
Di Zenzo-Cumani color edge
operators (Alg. 16.2). The
original color image (a) has
constant luminance, that is,
the intensity gradient is zero
and thus a simple grayscale
operator would not detect
any edges at all. The local
edge strength E(u) is almost
identical for both color edge
operators (b). Edge tangent
orientation vectors (normal to
(a) (b) Φ(u)) for the monochromatic
and multi-gradient operators
Monochromatic operator Di Zenzo-Cumani operator (c, d); enlarged details in (e, f).

(c) (d)

(e) (f)


maximum edge strength λ1 (u) = ∇I(u)2 is equivalent to the
magnitude of the local intensity gradient.13 The fact that λ2 = 0
indicates that the local contrast in the orthogonal direction (i.e.,
along the edge tangent) is zero (see Fig. 16.3(c)).
To calculate the local edge orientation, at position u we use Eqn.
(16.31) to get
13
See Eqns. (6.5) and (6.13) in Chapter 6, Sec. 6.2.
403
 16 Edge Detection in
Color Images

Fig. 16.5
Results of Di Zenzo-Cumani
color edge operator (Alg.
16.2) on real images. Orig-
inal image (a) and inverted
color edge magnitude (b).
The images in (c) show
the differences to the edge (a)
magnitude returned by the
monochromatic operator (Alg.
16.1, using the L∞ norm).

(b)

(c)

2C 2C Ix (u) · Iy (u) Iy (u)

tan(θ1 (u)) = = = =
A − B + (A + B) 2A Ix2 (u) Ix (u)
(16.39)

and the direction of maximum contrast14 is then found as

 I (u) 
y
θ1 (u) = tan−1 = ArcTan(Ix (u), Iy (u)). (16.40)
Ix (u)
Thus, for scalar-valued images, the general (multi-dimensional) tech-
nique based on the eigenvalues of the structure matrix leads to ex-
actly the same result as the conventional grayscale edge detection
approach described in Chapter 6, Sec. 6.3.

16.3 Canny Edge Detector for Color Images

Like most other edge operators, the Canny detector was originally
designed for grayscale (i.e., scalar-valued) images. To use it on color
images, a trivial approach is to apply the monochromatic operator
separately to each of the color channels and subsequently merge the
results into a single edge map. However, since edges within the dif-
ferent color channels rarely occur in the same places, the result will
14
See Eqn. (6.14) in Chapter 6.
404
usually contain multiple edge marks and undesirable clutter (see Fig. 16.3 Canny Edge
16.8 for an example). Detector for Color
Fortunately, the original grayscale version of the Canny edge de- Images
tector can be easily adapted to color imagery using the multi-gradient
concept described in Sec. 16.2.1. The only changes required in Alg.
6.1 are the calculation of the local gradients and the edge magnitude
Emag . The modified procedure is shown in Alg. 16.3 (see Sec. 16.5
for the corresponding Java implementation).

Alg. 16.3
1: ColorCannyEdgeDetector(I, σ, thi , tlo ) Canny edge detector for color
Input: I = (IR , IG , IB ), an RGB color image of size M × N ; images. Structure and pa-
σ, radius of Gaussian filter H G,σ ; thi , tlo , hysteresis thresholds rameters are identical to the
grayscale version in Alg. 6.1
(thi > tlo ). Returns a binary edge image of size M ×N . (p. 135). In the algorithm be-
low, edge magnitude (Emag )
2: I¯R ← IR ∗ H G,σ ⊲ blur components with Gaussian of width σ and orientation (Ex , Ey ) are
3: I¯G ← IG ∗ H G,σ obtained from the gradients of
4: I¯B ← IB ∗ H G,σ the individual color channels
(as described in Sec. 16.2.1).
5: Hx∇ ← [−0.5 0 0.5 ] ⊲ x gradient filter
⊺
6: Hy∇ ← [−0.5 0 0.5 ] ⊲ y gradient filter
7: I¯R,x ← I¯R ∗ Hx∇ , I¯R,y ← I¯R ∗ Hy∇
8: I¯G,x ← I¯G ∗ Hx∇ , I¯G,y ← I¯G ∗ Hy∇
9: I¯B,x ← I¯B ∗ Hx∇ , I¯B,y ← I¯B ∗ Hy∇
10: (M, N ) ← Size(I)
11: Create maps:
12: Emag , Enms , Ex , Ey : M ×N → R
13: Ebin : M ×N → {0, 1}
14: for all image coordinates u ∈ M ×N do
15: (rx , gx , bx ) ← (IR,x (u), IG,x (u), IB,x (u))
16: (ry , gy , by ) ← (IR,y (u), IG,y (u), IB,y (u))
17: A ← rx2 + gx2 + b2x ,
18: B ← ry2 + gy2 + b2y
19: C ← rx ·ry + gx ·gy + bx ·by
1/2
20: D ← (A−B)2 + 4C 2
 1/2 √
21: Emag (u) ← 0.5 · A + B + D ⊲ λ1 , Eq. 16.27
22: Ex (u) ← A − B + D ⊲ q 1 , Eq. 16.28
23: Ey (u) ← 2C
24: Enms (u) ← 0
25: Ebin (u) ← 0
26: for u ← 1, . . . , M −2 do
27: for v ← 1, . . . , N −2 do
28: u ← (u, v)
29: dx ← Ex (u)
30: dy ← Ey (u)
31: s ← GetOrientationSector(dx , dy ) ⊲ Alg. 6.2
32: if IsLocalMax(Emag , u, s, tlo ) then ⊲ Alg. 6.2
33: Enms (u) ← Emag (u)
34: for u ← 1, . . . , M −2 do
35: for v ← 1, . . . , N −2 do
36: u ← (u, v)
37: if (Enms (u) ≥ thi ∧ Ebin (u) = 0) then
38: TraceAndThreshold(Enms , Ebin , u, v, tlo ) ⊲ Alg. 6.2
39: return Ebin .
405
16 Edge Detection in In the pre-processing step, each of the three color channels is
Color Images individually smoothed by a Gaussian filter of width σ, before cal-
culating the gradient vectors (Alg. 16.3, lines 2–9). As in Alg. 16.2,
the color edge magnitude is calculated as the squared local contrast,
obtained from the dominant eigenvalue of the structure matrix M
(Eqns. (16.24)–(16.27)). The local gradient vector (Ex , Ey ) is cal-
culated from the elements A, B, C, of the matrix M, as given in
Eqn. (16.28). The corresponding steps are found in Alg. 16.3, lines
14–22. The remaining steps, including non-maximum suppression,
edge tracing and thresholding, are exactly the same as in Alg. 6.1.
Results from the grayscale and color version of the Canny edge
detector are compared in Figs. 16.6 and 16.7 for varying values of
σ and thi , respectively. In all cases, the gradient magnitude was
normalized and the threshold values thi , tlo are given as a percent-
age of the maximum edge magnitude. Evidently, the color detector
gives the more consistent results, particularly at color edges with low
intensity difference.
For comparison, Fig. 16.8 shows the results of applying the
monochromatic Canny operator separately to each color channel and
subsequently merging the edge pixels into a combined edge map, as
mentioned at the beginning of this section. We see that this leads
to multiple responses and cluttered edges, since maximum gradient
positions in the different color channels are generally not collocated.
In summary, the Canny edge detector is superior to simpler
schemes based on first-order gradients and global thresholding, in
terms of extracting clean and well-located edges that are immedi-
ately useful for subsequent processing. The results in Figs. 16.6 and
16.7 demonstrate that the use of color gives additional improvements
over the grayscale approach, since edges with insufficient brightness
gradients can still be detected from local color differences. Essential
for the good performance of the color Canny edge detector, how-
ever, is the reliable calculation of the gradient direction, based on
the multi-dimensional local contrast formulation given in Sec. 16.2.3.
Quite a few variations of Canny detectors for color images have been
proposed in the literature, including the one attributed to Kanade
(in [140]), which is similar to the algorithm described here.

16.4 Other Color Edge Operators

The idea of using a vector field model in the context of color edge
detection was first presented by Di Zenzo [63], who suggested finding
the orientation of maximum change by maximizing S(u, θ) in Eqn.
(16.18) over the angle θ. Later Cumani [59, 60] proposed directly
using the eigenvalues and eigenvectors of the local structure matrix
M (Eqn. (16.24)) for calculating edge strength and orientation. He
also proposed using the zero-crossings of the second-order gradients
along the direction of maximum contrast to precisely locate edges,
which is a general problem with first-order techniques. Both Di Zenzo
and Cumani used only the dominant eigenvalue, indicating the edge
strength perpendicular to the edge (if an edge existed at all), and then
discarded the smaller eigenvalue proportional to the edge strength in
406
Canny (grayscale) Canny (color)
16.4 Other Color Edge
Operators

Fig. 16.6
Canny grayscale vs. color
version. Results from the
grayscale (left) and the color
version (right) of the Canny
operator for different values of
σ (thi = 20%, tlo = 5% of max.
edge magnitude).

(a) (b)

(c) σ = 0.5 (d)

(e) σ = 1.0 (f)

(g) σ = 2.0 (h)

(i) σ = 5.0 (i)

407
 Canny (grayscale) Canny (color)
16 Edge Detection in
Color Images

Fig. 16.7
Canny grayscale vs. color
version. Results from the
grayscale (left) and the
color version (right) of the
Canny operator for different
threshold values thi , given
in % of max. edge magni-
tude (tlo = 5%, σ = 2.0).
(a) (b)

(c) thi = 10% (d)

(e) thi = 30% (f)

(g) thi = 50% (h)

(i) thi = 70% (i)

408
 σ = 2.0 σ = 5.0
16.4 Other Color Edge
Operators

Fig. 16.8
Scalar vs. vector-based color
Canny operator. Results from
the scalar Canny operator ap-
plied separately to each color
channel (a, b). Channel edges
are shown in corresponding
colors, with mixed colors indi-
cating that edge points were
(a) (b) detected in multiple channels
(e.g., yellow marks overlapping
points from the red and the
green channel). A black pixel
indicates that an edge point
was detected in all three color
channels. Channel edges com-
bined into a joint edge map
(c, d). For comparison, the re-
sult of the vector-based color
Canny operator (e, f). Com-
mon parameter settings are
σ = 2.0 and 5.0, thi = 20%,
tlo = 5% of max. edge magni-
(c) (d) tude.

(e) (f)

the perpendicular (i.e., tangential) direction. Real edges only exist

where the larger eigenvalue is considerably greater than the smaller
one. If both eigenvalues have similar values, this indicates that the
local image surface exhibits change in all directions, which is not
typically true at an edge but quite characteristic of flat, noisy regions
and corners. One solution therefore is to use the difference between
the eigenvalues, λ1 −λ2 , to quantify edge strength [206].
Several color versions of the Canny edge detector can be found in
the literature, such as the one proposed by Kanade (in [140]) which
is very similar to the algorithm presented here. Other approaches of
adapting the Canny detector for color images can be found in [85].
In addition to Canny’s scheme, other types of color edge detectors
have been used successfully, including techniques based on vector
order statistics and color difference vectors. Excellent surveys of
the various color edge detection approaches can be found in [266]
and [141, Ch. 6].

409
16 Edge Detection in 16.5 Java Implementation
Color Images
The following Java implementations of the algorithms described in
this chapter can be found in the source code section15 of the book’s
website. The common (abstract) super-class for all color edge de-
tectors is ColorEdgeDetector, which mainly provides the following
methods:
FloatProcessor getEdgeMagnitude ()
Returns the resulting edge magnitude map E(u) as a Float-
Processor object.
FloatProcessor getEdgeOrientation ()
Returns the resulting edge orientation map Φ(u) as a Float-
Processor object, with values in the range [−π, π].
The following edge detectors are defined as concrete sub-classes of
ColorEdgeDetector:
GrayscaleEdgeDetector: Implements an edge detector that uses
only the intensity (brightness) of the supplied color image.
MonochromaticEdgeDetector: Implements the monochromatic color
edge detector described in Alg. 16.1.
DiZenzoCumaniEdgeDetector: Implements the Di Zenzo-Cumani ty-
pe color edge detector described in Alg. 16.2.
CannyEdgeDetector: Implements the canny edge detector for gray-
scale and color images described in Alg. 16.3. This class defines
the additional methods
ByteProcessor getEdgeBinary(),
List<List<java.awt.Point>> getEdgeTraces().
Program 16.1 shows a complete example for the use of the class
CannyEdgeDetector in the context of an ImageJ plugin.

15
Package imagingbook.pub.color.edge.
410
 1 import ij.ImagePlus; 16.5 Java
2 import ij.plugin.filter.PlugInFilter; Implementation
3 import ij.process.ByteProcessor;
4 import ij.process.FloatProcessor; Prog. 16.1
5 import ij.process.ImageProcessor; Use of the CannyEdgeDetector
6 import imagingbook.pub.coloredge.CannyEdgeDetector; class in an ImageJ plugin. A
parameter object (params) is
7 created in line 20, subsequently
8 import java.awt.Point; configured (in lines 22–24)
and finally used to construct
9 import java.util.List; a CannyEdgeDetector object in
10 line 27. Note that edge detec-
11 public class Canny_Edge_Demo implements PlugInFilter { tion is performed within the
constructor method. Lines 29–
12 33 demonstrate how different
13 public int setup(String arg0, ImagePlus imp) { types of edge detection results
14 return DOES_ALL + NO_CHANGES; can be retrieved. The binary
edge map eBin is displayed in
15 } line 35. As indicated in the
16 setup() method (by returning
DOES_ALL), this plugin works
17 public void run(ImageProcessor ip) { with any type of image.
18
19 CannyEdgeDetector.Parameters params =
20 new CannyEdgeDetector.Parameters();
21
22 params.gSigma = 3.0f; // σ of Gaussian
23 params.hiThr = 20.0f; // 20% of max. edge magnitude
24 params.loThr = 5.0f; // 5% of max. edge magnitude
25
26 CannyEdgeDetector detector =
27 new CannyEdgeDetector(ip, params);
28
29 FloatProcessor eMag = detector.getEdgeMagnitude();
30 FloatProcessor eOrt = detector.getEdgeOrientation();
31 ByteProcessor eBin = detector.getEdgeBinary();
32 List<List<Point>> edgeTraces =
33 detector.getEdgeTraces();
34
35 (new ImagePlus("Canny Edges", eBin)).show();
36
37 // process edge detection results ...
38 }
39 }

411