Lecture 03

Image Processing,
Retrieval, and Analysis (I)

Prof. Christian Bauckhage
Outline
Lecture 03
Recap
Representation of Digital Images (II)

Images as Sets
Images as Arrays
Images as Functions
Derivatives of Images
Emboss Effect
Summary
Exercises
Recap
recording technology for digital (color-)photography

I intensity images are recorded using CCD chips that
consist of many photosensitive elements
CCD chip schematic CCD matrix

Recap

I color images result from splitting light beams into basic
components . . .
trichroic prism three separate CCD arrays

Recap

I . . . or from filtering different wavelengths prior to recording
Bayer filter single array

Recap
digitization and quantization

I the spatial resolution of a digital image is characterized by
its width and height
I typically, images are stored using a maximum of 256 = 28
intensity levels per channel
I this requires 8 bit (= 1 byte) per CCD element
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
1 2 3 4 5
I digital images are digitized and quantized

Images as Sets
images as pixel sets

I in an abstract manner, we may think of an intensity image
as a set

I = pij pij ∈ {0, . . . , 255} ∧ i ∈ {0, . . . , rmax } ∧ j ∈ {0, . . . , cmax }

Images as Sets
images as pixel sets

I in an abstract manner, we may think of an intensity image
as a set

I = pij pij ∈ {0, . . . , 255} ∧ i ∈ {0, . . . , rmax } ∧ j ∈ {0, . . . , cmax }

note:
I this view is particularly appropriate for binary images

where p ∈ {0, 1}
Images as Arrays
images as n-way pixel arrays

I the domain of digital images is discrete
I in computer memory, they are typically

represented as n-way arrays of pixels
Images as Arrays
1-way array (“vector”)
2-way array (“matrix”) 3-way array (“tensor”)

Images as Arrays
intensity image (“matrix”)
color image (“tensor”)

Images as Arrays
pixels and pixel arrays

I in a 2D array, a pixel p
is addressed using i pij
its coordinates (i, j)

I we also write pij
j
Images as Arrays
pixels and pixel arrays

I in a 2D array, a pixel p
is addressed using i pij
its coordinates (i, j)

I we also write pij
note:
I i is the row index and j is the column index and, following

the conventions for matrices or tensors, coordinate (0, 0)
is located in the upper left corner of the image
Images as Arrays
intensity images as 2- or 1-way arrays

I pixel p can either be addressed trough
two indices i and j denoting its row and
column coordinates and we write pij . . .
Images as Arrays

I . . . or through a single index k and we write pk

Images as Arrays

I . . . or through a single index k and we write pk
I using jmax = width of a digital image, we have the following

relationship
pij ⇔ p(i·jmax +j) = pk

Images as Arrays
pixel neighborhoods
I in image processing, one often considers the
4-neighborhood and the 8-neighborhood of a
pixel p
p p
4-neighborhood 8-neighborhood
Images as Arrays
pixel neighborhoods
I generally, the spatial distance between two pixels p and q
with coordinates (ip , jp ) and (iq , jq ) is measured using the
Euclidean distance between their coordinates
q
d(p, q) = (ip , jp ) − (iq , jq ) = (ip − iq )2 + (jp − jq )2

Images as Arrays
pixel neighborhoods
I for the pixels q in the 4-neighborhood of p, the Euclidean
distance between p and q amounts to
d(p, q) = 1
Images as Arrays
pixel neighborhoods
I for the pixels q in the 4-neighborhood of p, the Euclidean
distance between p and q amounts to
d(p, q) = 1
I for the pixels in the 8-neighborhood of a pixel p, this does

not always hold; for instance
q
2 2
d(pij , pi+1j+1 ) = i − (i + 1) + j − (j + 1)
p
= 12 + 12
√
= 2
Images as Arrays
Question:
I can’t we consider or define a distance that would
avoid such “strange behavior”?
Images as Arrays
Question:
I can’t we consider or define a distance that would
avoid such “strange behavior”?
Answer:
I yes, we can! (more on this later!)
I but we rather shouldn’t (more on this later!)
Images as Arrays
Question:
I why does image processing resorts to pixel arrays where
some immediate neighbors are closer than others?
I is it because it is impossible to find an arrangement where

pixels cover the image plane and have an equal distance
to all their immediate neighbors?
Images as Arrays
Question:
I why does image processing resorts to pixel arrays where
some immediate neighbors are closer than others?
I is it because it is impossible to find an arrangement where

pixels cover the image plane and have an equal distance
to all their immediate neighbors?
Answer:
I no, in principle, there are many possibilities to achieve this!
Images as Arrays
tessellation / tiling
I a tessellation is a collection of planar figures that cover
the entire plane without overlaps and gaps
I examples:
Images as Arrays
I interesting tessellations have certain symmetry properties,
they are periodic or a-periodic, . . .
I most notably, there are tessellations where the Euclidean

distance between the centers of adjacent elements is
always the same
Images as Arrays
I interesting tessellations have certain symmetry properties,
they are periodic or a-periodic, . . .
I most notably, there are tessellations where the Euclidean

distance between the centers of adjacent elements is
always the same
Question:
I then why are these not used in digital photography?
curSubSection
Answer:
I the reason why these are usually neglected in
digital imaging is of rather technical nature!
curSubSection
Answer:
I as of today, the memory of computers is typically

conceptualized as a (very long) linear sequence
of storage cells
curSubSection
Answer:
I as of today, the memory of computers is typically

conceptualized as a (very long) linear sequence
of storage cells
I therefore, the simplest approach is to represent

the rows of an image as a sequence of storage
cells and to represent the image as a sequence
of such rows, i.e. as a 1-way pixel array
Images as Functions
intensity images as discrete functions

I in a gray value image, discrete coordinates are mapped
to discrete intensity values
I a gray value image may therefore also be thought of as a

2D discrete function g[i, j] where

g : 0, . . . , cmax × 0, . . . , rmax → 0, . . . , 255
Images as Functions
intensity images as discrete functions

I in a gray value image, discrete coordinates are mapped
to discrete intensity values
I a gray value image may therefore also be thought of as a

2D discrete function g[i, j] where

g : 0, . . . , cmax × 0, . . . , rmax → 0, . . . , 255
note:
I in contrast to the matrix interpretation of an image, here,

index i denotes image columns, j denotes image rows,
and coordinate [0, 0] is located in the lower left corner
Images as Functions
example
50
45
40
35
30
25
20
15
150 10
5
0 5 10 15 20 25 0
30 35 40 45 50
Images as Functions
intensity images as partial functions

I assuming a slightly more abstract point of view, g[i, j] can
also be thought of as a partial function

g : R2 0, . . . , 255
Images as Functions


g : R2 0, . . . , 255
I tuples (x, y) ∈ R2 are mapped to gray

i.e. while certain
values in the set 0, . . . , 255 many (or most) possible
tuples will not be assigned a value
Images as Functions


g : R2 0, . . . , 255
I tuples (x, y) ∈ R2 are mapped to gray

i.e. while certain
values in the set 0, . . . , 255 many (or most) possible
tuples will not be assigned a value
note:
I instead of g[i, j] we also write g[x], x ∈ R2 to express that

we understand an image as a partial, 2D function
Images as Functions
color images
I in a color image, discrete coordinates are mapped to
certain color values which are represented
by triples
of numbers taken from the set 0, . . . , 255
I a color image thus is a vector valued function
I a color image can thus be seen as a vector field c[x]

where
3
c : R2 0, . . . , 255
Images as Functions
Question:
I what do we gain from raising the level of abstraction?
Images as Functions
Question:
Answer:
I if we are dealing with mathematical functions, we may
apply the whole toolbox of mathematical techniques
which have been developed over centuries
I we can apply functions or operators to images, i.e. we
can compute with images
Images as Functions
Question:
Answer:
I if we are dealing with mathematical functions, we may
apply the whole toolbox of mathematical techniques
which have been developed over centuries
I we can apply functions or operators to images, i.e. we
can compute with images
this is the basis of image processing

let’s look at an example
derivative of a continuous 1D function f (x)

I derivative from above
f (x + ε) − f (x)
f+0 (x) = lim
ε→0 ε
I derivative from below
f (x) − f (x − ε)
f−0 (x) = lim
ε→0 ε
derivative of a continuous 1D function f (x)

f (x + ε) − f (x)
f+0 (x) = lim
ε→0 ε
f (x) − f (x − ε)
f−0 (x) = lim
ε→0 ε
I f (x) is differentiable in x if
d
f+0 (x) = f−0 (x) = f 0 (x) = f (x)
dx
example

I derivatives from above and from below of f (x) = x2 x=2
16 16 16 16
12 12 12 12
8 8 8 8
f+0 : 4 4 4 ... 4
0 0 0 0
-2 0 2 4 -2 0 2 4 -2 0 2 4 -2 0 2 4
ε=2 ε=1 ε = 0.5 lim ε → 0

16 16 16 16
12 12 12 12
8 8 8
8
f−0 : 4
4 4 ... 4
0 0 0
0 -2 0 2 4 -2 0 2 4 -2 0 2 4
-2 0 2 4
ε=2 ε=1 ε = 0.5 lim ε → 0

note:
I f (x) may be continuous in x and f+0 (x) and f−0 (x)

may exists, but they need not be identical
I f (x) may not be differentiable in x

derivative of a discrete 1D function f [i]

f [i + 1] − f [i]
∆+ f [i] = = f [i + 1] − f [i]
i+1−i
f [i] − f [i − 1]
∆− f [i] = = f [i] − f [i − 1]
i − (i − 1)
derivative of a discrete 1D function f [i]

f [i + 1] − f [i]
∆+ f [i] = = f [i + 1] − f [i]
i+1−i
f [i] − f [i − 1]
∆− f [i] = = f [i] − f [i − 1]
i − (i − 1)
I because of the discrete nature of f [i], it is impossible

to consider a limit process
⇒ the minimum of the denominator will always be 1
⇒ the derivative from above and the derivative from
below will generally differ
example

I derivatives from above and from below of f [i] = i2 i=2
16 16
12 12
8 8
4 4
0 0
0 1 2 3 4 0 1 2 3 4
∆+ f [2] ∆− f [2]
“compromise”
I when computing the derivative of a discrete function,
consider information from below and from above:
∆f [i] = f [i + 1] − f [i − 1]
example

I discrete derivative of f [i] = i2 i=2
16
12
0
0 1 2 3 4
∆f [2]
gradient
I the gradient of a multivariate continuous function
f (x1 , x2 , . . . , xn ) is the vector of partial derivatives of f
 ∂f 
∂x1
∇f (x1 , . . . , xn ) =  ... 
 
∂f
∂xn
gradient
I the gradient of a multivariate continuous function
f (x1 , x2 , . . . , xn ) is the vector of partial derivatives of f
 ∂f 
∂x1
∇f (x1 , . . . , xn ) =  ... 
 
∂f
∂xn
I accordingly, we can define a gradient of an intensity

image g[i, j] by requiring

∆x g[i, j] g[i + 1, j] − g[i − 1, j]
∇g[i, j] = =
∆y g[i, j] g[i, j + 1] − g[i, j − 1]
note:
I the gradient is an operator
∇ : (Rn → R) → (Rn → Rn )
note:
∇ : (Rn → R) → (Rn → Rn )
I that is, it is a function that operates on functions

note:
∇ : (Rn → R) → (Rn → Rn )
I that is, it takes a function and produces another function

note:
∇ : (Rn → R) → (Rn → Rn )
I that is, it takes a function and produces another function
I contrast this with functions which take a (tuple of)

number(s) and produce another (tuple of) number(s)
gradient
I ∇g[i, j] is a discrete, vector valued function, i.e. a discrete
vector field
gradient
vector field
I for
visualization,
one usually considers only the norm
∇g[i, j] of ∇g[i, j], where
q
q
2
∇g[i, j] = ∇g[i, j], ∇g[i, j] = ∇g[i, j]
gradient
vector field
I for
visualization,
q
q
2
∇g[i, j] = ∇g[i, j], ∇g[i, j] = ∇g[i, j]
I also, values exceeding 255 are typically rounded off to 255

gradient
vector field
I for
visualization,
q
q
2
∇g[i, j] = ∇g[i, j], ∇g[i, j] = ∇g[i, j]
I also, values exceeding 255 are typically rounded off to 255
I sometimes,
one only plots those coordinates (i, j), where
∇g[i, j] assumes a local maximum
example
g[i, j] ∆x g[i, j]

∆y g[i, j] ∇g[i, j]
example
450
400
350
300
250
200
150
100
50
0 100 200 300 400 500 600 700
∇g[i, j] shown as a vector field

Question:
I what is this good for?
Question:
Answer:
I it detects edges!
Question:
Answer:
I it detects edges!
Question:
Question:
Answer:
I it detects edges!
Question:
Answer:
I we will study numerous applications of this later
I for now, let’s consider an example from artistic
image processing
Emboss Effect
idea
I transform an input image such that it looks like a
copper engraving or etching
Emboss Effect
idea
I transform an input image such that it looks like a
copper engraving or etching
approach
I similar to the computation of derivatives
I transform image g into g̃ using

g̃[i, j] = 128 + g[i + 1, j + 1] − g[i − 1, j − 1]
Emboss Effect
examples
Summary
we now know about

I mathematical representations of digital images
I continuous derivatives and gradients
I discrete derivatives and gradients

Exercises
1. the divergence of a continuous multivariate function

f (x1 , x2 , . . . , xn ) is defined as
X
n
∂2 f
∆f = ∇2 f = ∇ · ∇f =
i=1
∂xi2
where ∆ is called the Laplace operator; write down how to

compute the divergence of a discrete 2D function g[i, j]
2. assume a discrete 1D function g[i]; what does the following

operator do?
1
AM g[i] = g[i − 1] + g[i] + g[i + 1]
3
what about this one?
1
WM g[i] = g[i − 1] + 2g[i] + g[i + 1]
4

Lecture 03

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lecture 03

Uploaded by

Copyright:

Available Formats

Image Processing,

Retrieval, and Analysis (I)

Representation of Digital Images (II)

recording technology for digital (color-)photography

CCD chip schematic CCD matrix

recording technology for digital (color-)photography

trichroic prism three separate CCD arrays

recording technology for digital (color-)photography

Bayer filter single array

digitization and quantization

I digital images are digitized and quantized

images as pixel sets

images as pixel sets

I this view is particularly appropriate for binary images

images as n-way pixel arrays

I in computer memory, they are typically

1-way array (“vector”)

2-way array (“matrix”) 3-way array (“tensor”)

intensity image (“matrix”)

color image (“tensor”)

pixels and pixel arrays

its coordinates (i, j)

pixels and pixel arrays

its coordinates (i, j)

I i is the row index and j is the column index and, following

intensity images as 2- or 1-way arrays

intensity images as 2- or 1-way arrays

I . . . or through a single index k and we write pk

intensity images as 2- or 1-way arrays

I . . . or through a single index k and we write pk

I using jmax = width of a digital image, we have the following

pij ⇔ p(i·jmax +j) = pk

I for the pixels in the 8-neighborhood of a pixel p, this does

I is it because it is impossible to find an arrangement where

I is it because it is impossible to find an arrangement where

I most notably, there are tessellations where the Euclidean

I most notably, there are tessellations where the Euclidean

I as of today, the memory of computers is typically

I as of today, the memory of computers is typically

I therefore, the simplest approach is to represent

intensity images as discrete functions

I a gray value image may therefore also be thought of as a

intensity images as discrete functions

I a gray value image may therefore also be thought of as a

I in contrast to the matrix interpretation of an image, here,

intensity images as partial functions

intensity images as partial functions

I tuples (x, y) ∈ R2 are mapped to gray

intensity images as partial functions

I tuples (x, y) ∈ R2 are mapped to gray

I instead of g[i, j] we also write g[x], x ∈ R2 to express that

I a color image thus is a vector valued function

I a color image can thus be seen as a vector field c[x]

this is the basis of image processing

derivative of a continuous 1D function f (x)

derivative of a continuous 1D function f (x)

ε=2 ε=1 ε = 0.5 lim ε → 0

ε=2 ε=1 ε = 0.5 lim ε → 0

I f (x) may be continuous in x and f+0 (x) and f−0 (x)

I f (x) may not be differentiable in x

derivative of a discrete 1D function f [i]

derivative of a discrete 1D function f [i]

I because of the discrete nature of f [i], it is impossible