Journal of Visual Communication and Image Representation 11, 237244 (2000)

doi:10.1006/jvci.1999.0446, available online at http://www.idealibrary.com on

Blur and Disorder

Jan J. Koenderink and Andrea J. van Doorn
Department of Physics and Astronomy, Universiteit Utrecht, PO Box 80 000, 3508TA Utrecht, The Netherlands

Received September 28, 1999; accepted November 14, 1999

Blurring is not the only way to selectively remove fine spatial detail from an
image. An alternative is to scramble pixels locally over areas defined by the desired
blur circle. We refer to such scrambled images as locally disorderly. Such images
have many potentially interesting applications. In this contribution we discuss a
formal framework for such locally disorderly images. It boils down to a number of
intricately intertwined scale spaces, one of which is the ordinary linear scale space
for the image. The formalism is constructed on the basis of an operational definition
of local histograms of arbitrary bin width and arbitrary support. C 2000 Academic Press

1. INTRODUCTION

Standing in front of a tree you may believe that you clearly see the leaves. You also see
foliage (which is a kind of texture), the general shape of the treetop, and so forth [1, 2].
You will be hard put to keep a given leaf in mind (say) and retrieve it the next day, or even
a minute later. You fail any idea of the total number of leaves. If you glance away and we
would remove a few leaves you would hardly notice. Indeed, it seems likely that you would
not notice quite serious changes in the foliage at all: One branch tends to look much like any
other. This is what Ruskin [3, 4] called mystery. Yet you would notice when the treetop
was replaced with a balloon of the same shape and identical average color. The foliage
texture is important (it makes the foliage look like what it is) even though the actual spatial
structure is largely ineffective in human vision.
Something quite similar occurs with images. Given an image we may scramble its pixels
or replace all pixels with the average taken over all pixels. Both operations clearly destroy all
spatial information. Yet the scrambled picture contains more information than the blurred
one. When the blurring or scrambling is done locally at many places in a large image,
one obtains something like a blurred photograph in one case, like a painterly rendering
in the other case. Although the painterly rendering does not reveal more spatial detail
than the blurred one, it tells you more about the rendered scene. We call such images
locally disorderly. We became interested because human vision is locally disordered in the
peripheral visual field. In pathological cases it may even dominate focal vision [5, 6]. For
instance, in cases of (always unilateral) scrambled vision the visual field is useless for
reading newspapers (even the headlines) say. Yet such patients can discriminate even the
237

1047-3203/00 $35.00
Copyright C 2000 by Academic Press
All rights of reproduction in any form reserved.
238 KOENDERINK AND VAN DOORN

highest spatial frequencies from a uniform field: In that respect the scrambled eye is at
par with the good one. There is nothing wrong with the optics of such eyes, nor with the
sampling. The patients simply do not know where the pixels are; it is a mental deficiency,
not an optical or a neurophysiological disorder.
Apart from the natural urge to understand such locally disorderly images formally, they
may have many applications in computer graphics (why compute all leaves on a tree when
a painter can do without?), image compression (why encode what the human observer does
not notice?), and so forth. However, our interest is mainly fundamental.

2. LOCAL HISTOGRAMS

For an image showing an extensive scene the histogram of the full image makes little
sense because it is a hodge-podge of mutually unrelated entities. Far more informative
are histograms of selected regions of interest that show certain uniform regions (where the
precise concept of uniformity may vary). Such a region may be part of a treetop for instance.
A pixel histogram still might show quite different structure according to the size of the leaf
images with respect to the pixel size. In order to understand the structure of the histogram
one has to specify both the region of interest and the spatial resolution. Finally, the result
will depend upon the resolution in the intensity domain: The histograms for a binary image,
an 8 bit image, and a 32 bit image will all look different. We will refer to this as the bin
width. Thus one needs at least the following parameters to specify a local histogram: The
location and size of the region of interest (the support), the spatial resolution, and the bin
width (that is, the resolution in the intensity domain). We will regard the histogram as a
function of position; thus one obtains a histogram valued image.
In order to construct histogram valued images one has to measure how much stuff
should go in a certain bin for a certain region of interest, at a certain resolution. The spatial
distribution of this stuff is what makes up the histogram valued image for a certain region
size, resolution, bin width, and intensity value. For each intensity value (keeping the other
parameters constant) one has such a stuff distribution image.
Here is how one might construct a stuff distribution image: First one constructs the image
at the specified level of resolution. Formally [79] this can be written as a convolution (here
denotes the convolution operator)

I (r; ) = S(r) G0 (r; ) (1)

of the scene (S(r); at infinite resolution) with a Gaussian kernel

e 2 2
r;r

G 0 (r; ) = , (2)
2 2
where denotes the resolution. Of course this is to be understood as merely formal: In
reality one observes the image; the scene is implicitit cannot be observed other than via
images. It is like the horizon that can never be reached.
Next one applies the following nonlinear transformation to the intensity of each pixel,

F(i; i 0 , ) = e(ii0 ) /2 ,
2 2
(3)

where i 0 denotes the fiducial intensity value and denotes the bin width. Notice that the
resulting image has pixel values in the range zero to one. These values can be regarded as
the value of the membership function of a given pixel to the specified bin. Finally one blurs
 BLUR AND DISORDER 239

FIG. 1. An image and three distributions of histogram stuff. Fiducial level is 130, bin widths are 2, 8, and 32
(intensity ranges from 0 to 255). Notice that though the location of the soft isophote is constant, its width increases
as the bin width is increased. The soft isophotes can be quite broad ribbons; in fact there is no limit, they may fill
the whole image.

the result, taking the size of the region of interest as internal scale. The kernel to be used is

A(r; r0 , ) = e(rr0 ) (rr0 )/2 .

2
(4)

The function A is the aperture function which defines the (soft) region of interest. It is not
normalized; thus the total amount of stuff that gets into a bin will be proportional with the
area of the aperture. This distributes the membership over the region of interest. Thus the
local histograms are defined as
Z 2
1 (I (r; )i)
H (i; r0 , , , ) = A(r; r0 , )e 2 2
dr. (5)
2 2 image

Here we have normalized with respect to the area of the region of interest. The resulting
image will have an overall level zero with a certain diffuse ribbon (or curve-like) object that
may reveal values up to one. This ribbon is intuitively a representation of a soft isophote. It
is in many respects a more useful object than a true isophote. For instance, in regions where
the intensity is roughly constant the true isophotes are ill defined and often take on a fractal
character. The smallest perturbation may change their nature completely. In such regions
the ribbon spreads out and fills the roughly uniform region with a nonzero but low value:
The membership is divided over the pixels in the uniform region. For a linear gradient, the
steeper the gradient, the narrower the isophote. (See Figs. 13.)
Notice that we deal with a number of closely intertwined scale spaces here. First there
is the regular scale space of the image at several levels of resolution. Here the extentional
parameter is r and the scale parameter . Then there is the scale space of histograms. Here
the extentional parameter is the intensity i whereas the resolution parameter is the bin width.

FIG. 2. Four distributions of histogram stuff. The image is shown in Fig. 1 on the left. The bin width is 4,
fiducial levels are 90, 110, 130, and 150. Notice the shift of the isophote with level. In this respect the soft isophotes
behave exactly like regular isophotes (Fig. 3).
240 KOENDERINK AND VAN DOORN

FIG. 3. Four regular isophotes (at levels 90, 110, 130, and 150) of the image shown in Fig. 1 on the left.
Notice that the isophotes are not smooth curves but have a fractal character. Compare with the soft isophotes in
Fig. 2 for the same fiducial levels.

It is indeed (by construction) a neat linear scale space. One may think of it as the histogram
at maximum resolution, blurred with a Gaussian kernel at the bin width. Thus each pixel
in the image may be thought of as contributing a delta pulse at its intensity; these are then
blurred to obtain the histogram. This is formally identical to the Parzen estimator [10] of
the histogram. Finally, there is the scale space of stuff images with the size of the region of
interest as inner scale. The extentional parameter is r0 and the scale parameter .

3. LOCALLY DISORDERLY IMAGES

Consider an image and its histogram (in the classical sense, no scale space). When we do
arbitrary permutations on the pixels we obtain images that look quite distinct (the spatial
structure can differ in arbitrary ways), yet they all have the same histogram. Thus the his-
togram can be regarded as the image modulo its spatial structure. That it why we refer to such
images as disorderly in the first place. Histograms are zeroth order summary descriptions of
images, the conceptually simplest type of texture. A histogram valued image is likewise to
be considered locally disorderly. Notice that a disorderly image is not completely lacking
in spatial discrimination. For instance, consider an image that consists of vertical stripes
at the maximum frequency, say pixels with even horizontal index white, with odd index
black. When this image is scrambled it can still be discriminated from a uniform image that
is uniformly 50% gray. On the other hand, this discrimination fails when we completely
blur the image, that is say, replace every pixel value with the average over all pixels. Yet
the rudimentary spatial discrimination is not perfect; for instance, one cannot discriminate
between vertical and horizontal stripes.
Quite apart from spatial discrimination, locally disorderly images retain much informa-
tion that exists at a scale that is not resolved and would be lost in a totally blurred image [11,
12]. The type of information that is retained often lets you recognize material properties
that would be lost by total blurring. Locally disorderly images are very similar to painterly
renderings. In an impressionist painting there is often no spatial information on the scale of
the brush strokes (touches). At this scale one has structure that reveals the facture, often a
signature of the artist. Yet the strokes contribute much to the realistic rendering. The artist
does not paint leaves when he or she paints a treetop, he or she does foliage. Although no
leaves are ever painted you can often recognize the genus by the way the foliage is handled.
Pointillists were quite explicit about such things. For instance, Seurat [13] lists the colors
of touches that go in a rendering of grass. He never paints leaves of grass, yet the grass is
not uniformly green: There are blues (shadow areas), yellows (translucency), and oranges
(glints of sunlight). These touches appear in a locally disorderly pattern.
 BLUR AND DISORDER 241

FIG. 4. (Left) Original image. (Middle) Blurred image. (Right) Locally disordered image. The blur and local
disorder destroy roughly the same spatial detail. Notice that the locally disordered image looks better than the
blurred one and indeed retains more information, for instance the bright pixels of the specularity on the nose or
the necklace.

Locally disorderly images can be rendered (see Fig. 4) by using the histograms as densities
for a random generator. Every instance will be different; their totality is the locally disorderly
image. Although all instances are different, they look the same. Indeed, they are the same on
the scale where disorder gives way to order, and on finer scales the histograms are identical.
Such renderings are not unlike dithered representations [14, 15].

4. GENERALIZATIONS

In this paper we have considered histograms of pixel values. It is, of course, an obvious
generalization to include local histograms of derivatives or (better) differential invariants.
Then one essentially considers local histograms of the local jets up to some order [16], or
perhaps the jets modulo some transformation, say a rotation or a leftright symmetry. Such
higher order locally disorderly images specify the distribution of textural qualities. Such
structures have indeed been used (albeit in an ad hoc fashion) to create instances of textures
given a single prototypical example. The local disorder is to be considered zeroth order
texture: The local image structure is summarized via the intensity values (zeroth order)
alone. Adding gradient information (histograms of the first order edge detectors) would
yield first order texture, and so forth.
The formal description of such structures appears straightforward and would indeed yield
a very attractive and (most likely) useful description of image structure.

5. DISORDERLY IMAGES AND SEGMENTATION

When the boundary between two uniform regions is blurred the pixels appear to lose
their origin. For instance, blurring the boundary between a uniformly red and a uniformly
green field yields a ribbon of yellow pixels that appear to belong to neither side. This has
been a major incentive for the construction of nonlinear blurring methods: The idea was to
blur areas, but not over edges. With locally disorderly images this dilemma is automatically
solved in an unexpected way. Although the boundary indeed becomes spatially less and less
defined, the individual pixels hold on to their origin. In the example given above no yellow
pixels are ever generated. One does not know where the red area ends and the green begins
because there are red pixels to be found in the green area and vice versa. But the pixels
themselves do not change their hue. In terms of segmentation the red and green areas may
be said to overlap in the boundary area.
242 KOENDERINK AND VAN DOORN

FIG. 5. The histogram of a patch straddling the body-shadow edge (over the tip of the nose, including part of
a check in light and part of one in shadow) is bimodal.

FIG. 6. Segments belonging to the major modes of the histogram shown in Fig. 5. These are the illuminated
and the shadow side of the face. Notice how the segments overlap over the ridge of the nose.

FIG. 7. An example of transparent segments. On the left is the original image. In the middle is the segment
belonging to the light stripes of the dress and on the right is that belonging to the dark stripes. Note that in the
latter case the dark hair is included in the segment; in the former case the face. In the body part the dress is (at this
resolution) simultaneously light and dark. In the disorderly representation the locations of the stripes are lost.
 BLUR AND DISORDER 243

FIG. 8. Blurred and locally scrambled versions of the striped dress image.

In Figs. 5 and 6 we show a segmentation of a face on the basis of major modes in the
local histograms. The light and dark halves of the face are segmented. There is some minor
overlap on the bridge of the nose. In Fig. 7 we show a more extreme case: Here the segments
belonging to two major modes are extensively overlapping. This example illustrates that
the locally disorderly images can be thought of as entertaining multiple intensity levels at
any given location in the image [11, 12]. For instance, one may take the major modes in
the local histograms as intensity values. When there exists only a single major mode the
representation is much like the conventional one (one intensity per pixel). But if there are
two or more major modes the advantage of the locally disorderly representation becomes
clear. In Fig. 7 the person apparently wears a dress that is light and dark at the same time! Of
course this occurs only when the stripes of the dress have been lost in the disorder. In such
cases the difference between the blurred and the locally scrambled versions of an image is
of course dramatic (Fig. 8).

6. CONCLUSION

Locally disorderly images admit a neat formal description in terms of a number of

intimately interlocked scale spaces. They can easily be rendered and can thus be used
in image compression. They have distinct advantages over blurred images when one is
interested in image segmentation. This is often desirable; for instance, one may treat the
blue sky as continuous even where it is partly occluded by numerous twigs when seen
through a tree top. Thus locally disorderly images may well find applications in image
processing and interpretation.

ACKNOWLEDGMENT

The authors thank Jysje Koenderink for standing model for Figs. 16.

REFERENCES

1. H. Helmholtz, Handbuch der physiologischen Optik, Voss, Hamburg/Leipzig, 1866.

2. W. Metzger, Gesetze des Sehens, Waldemar Kramer, Frankfurt, 1975.
244 KOENDERINK AND VAN DOORN

3. J. Ruskin, Elements of Drawing (first ed. 1857), George Allen, Sunnyside/Orpington, 1900.
4. J. Ruskin, Modern Painters, Vol. I, Estes, Boston, 1873.
5. R. Hess, Developmental sensory impairment: Amblyopia or tarachopia?, Human Neurobiol. 1, 1982, 1729.
6. J. J. Koenderink and A. J. van Doorn, Invariant features of contrast detection: An explanation in terms of
self-similar detector arrays, Biol. Cybernet. 30, 1978, 157167.
7. J. J. Koenderink, The structure of images, Biol. Cybernet. 50, 1984, 363370.
8. T. Lindeberg, Scale-space Theory in Computer Vision, Kluwer, Boston, 1994.
9. L. M. J. Florack, Image Structure, Kluwer, Dordrecht, 1997.
10. E. Parzen, On estimation of a probability density function and mode, Ann. Math. Statist. 33, 1962, 10651076.
11. A. J. Noest and J. J. Koenderink, Visual coherence despite transparency or partial occlusion, Perception 19,
1990, 384.
12. L. D. Griffin, Scale-imprecision space, Image Vision Comput. 15, 1997, 369398.
13. W. I. Homer, Seurat and the Science of Painting, MIT Press, Cambridge, MA, 1964.
14. R. A. Ulichney, Digital Halftoning, MIT Press, Cambridge, MA, 1987.
15. R. A. Ulichney, Dithering with blue noise, Proc. IEEE 76, 1988, 5679.
16. J. J. Koenderink and A. J. van Doorn, Generic neighborhood operators, IEEE Trans. Pattern Anal. Mach.
Intell. 14, 1992, 597605.

JAN J. KOENDERINK graduated in physics and mathematics in 1967 at Utrecht University and he was an
associate professor in experimental psychology at the Universiteit Groningen. Then in 1974, he returned to the
Universiteit Utrecht where he presently holds a chair in the Department of Physics and Astronomy. He founded
the Helmholtz Instituut in which multidisciplinary work in biology, medicine, physics, and computer science is
coordinated. He has received an honorific degree (D.Sc.) in medicine from the University of Leuven and is a
member of the Royal Netherlands Academy of Arts and Sciences. He participates in the editorial boards of a
number of scientific journals, scientific boards of international conferences, and scientific institutes. His research
interests include cognitive science, ecological physics, and machine intelligence.

ANDREA J. VAN DOORN graduated in physics and mathematics in 1971 at Utrecht University. She has
participated in research on vision at Groningen University and is now connected with the Helmholtz Instituut of
the Universiteit Utrecht where she works on various topics in visual psychophysics and modeling of visual functions
in humans. Recently she accepted a position as associate professor at the Technische Universiteit Delft, faculty of
Industrial Design. Her research interests include cognitive science, ecological physics, and machine intelligence.