Computer Vision, Image Processing and

Computational Photography
Lecture 2: Digital image processing

Ayelet Heimowitz

October, 2022

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Color images

A color image has 3 color channels.

Last week we discussed the RGB color space.

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Color images

A color image has 3 color channels.

Last week we discussed the RGB color space.

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

RGB color space - issue

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Solution: XYZ color space

2
x( )
y( )
z( )
1.5

0.5

0
400 450 500 550 600 650 700
[nm]

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

XYZ color space

Linear conversion from RGB color space,

    
X 0.49 0.31 0.20 R
Y  = 1 0.17697 0.81240 0.01063 G 
0.17697
Z 0 0.01 0.99 B

1
Sometimes the factor of 0.17697 is omitted.

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

L∗ a∗ b∗ color space

The L∗ a∗ b∗ color space is approximately perceptually uniform.

Conversion from XYZ to L∗ a∗ b∗ uses a reference point
 T
Xn Yn Zn and the function
( 1
t3 t > δ3
f (t) = t 2δ
.
3δ 2
+ 3 otherwise

Specifically,

Y
L∗ = 116 · f ( )
Yn
X Y
a∗ = 500 · [f ( ) − f ( )]
Xn Yn
Y Z
b∗ = 200 · [f ( ) − f ( )]
Yn Zn
Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra
Y’CbCr color space

Color space used for image and video compression.

Closely related to the older color spaces YIQ, YUV.

Y0
    
0.299 0.587 0.114 R
Cb  = −0.168736 −0.331264 0.5  G 
Cr 0.5 −0.418688 −0.081312 B

Note that, Matlab’s rbg2gray command returns the Y 0

channel as the grayscale image.

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Image processing

A digital image is a function I (x, y ).

x and y are the spatial coordinates.

The spatial coordinates x and y determine a unique location

in the image. This location is called a pixel.

The value I (x, y ) is the pixel value. This can be a scalar value
indicating intensity (for grayscale images) or a vector (for
color images).

When we process an image, we create a new image from some

input image or images.

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Point operations

The simplest type of image processing applies some function

to each pixel value in the input image to form the
corresponding pixel value in the output image.

Figure: Merced River, Yosemite Figure: Image after addition of

Valley, 1866 by Albert Bierstadt a constant

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Point operations

Figure: Image after subtraction Figure: Image after

of a constant multiplication by a scalar

What if we want to make the dark pixels brighter and the

bright pixels darker?

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Histogram - examples

A few thousand years ago, cryptography consisted of a

mapping between each letter in the alphabet to a single and
unique letter in the same alphabet.

This function can be a shift cipher, where the mapping is

A B C D E F G ... T U V W X Y Z
D E F G H J K ... W X Y Z A B C

Another cipher uses some bijective function, e.g.

A B C D E F G ... T U V W X Y Z
R L U E Z T A ... K M N O S W Y

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Histogram - examples

Say I have a dice, and would like to verify that it is a fair dice.
I could roll the dice many time and count the occurrences of
each result.
Graphically, this would look like

Figure: Probably fair dice Figure: Likely loaded dice

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Histogram - examples
If I had two die, I can roll both and sum,

Figure: Probably fair die

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Image histogram

The image histogram counts the number of occurrences of

each possible pixel value.

Alternatively, the histogram can count the number of

occurrences of pixel values in different ranges, e.g. 0 − 2,
3 − 5, etc.

That is, we make a bin of possible outcomes, and count the

number of times we observe these outcomes in the image.

When the histogram is normalized, we get an empirical

probability.

Viewing the image histogram can show us problems with the

image
Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra
Image histogram

Figure: The Rocky Mountains,

Lander’s Peak, Bierstadt. Figure: Histogram of grayscale
image.

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Histogram equalization
Let h be the histogram of an image. We assume the size of
the image is M × N, and the pixel values range from 0 to L.
The empirical probability of the value i is
number of occurrences of i
h(i) =
MN

We define the cumulative histogram c as follows,

i
X
c(i) = h(j) = c(i − 1) + h(i)
j=0

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Histogram equalization

Let I (x, y ) = k, then the corresponding pixel in the output

image will have a value of

f (k) = b(L − 1) · c(k)c

Figure: h. Figure: c.

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Histogram equalization

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Histogram equalization

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Image filtering

Point operations worked on each pixel individually,

The histogram ignored the location of each pixel.

However, usually, most (or all) neighboring pixels will be

similar.

Idea: let’s use neighborhood information.

Uses: denoise, smooth, sharpen, etc.

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Image filtering - idea

The linear filter will replace each pixel with a weighted sum
over the pixel values in some (small) neighborhood around the
pixel.
The weights are called the filter coefficients.
We define the kernel h as the matrix whose entries are the
filter coefficients. For example,
 
1/9 1/9 1/9
h = 1/9 1/9 1/9 .
1/9 1/9 1/9

The filtering operation is

J(x, y ) = I (x, y ) ∗ h

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Image filtering - idea

That is,
k
X k
X
J(x, y ) = I (x − m, y − n)h(m, n)
m=−k n=−k

Alternatively, the filtering can be done with a correlation

operator (that is, via cross-correlation)

k
X k
X
J(x, y ) = I (x + m, y + n)h(m, n)
m=−k n=−k

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Averaging filter

In the averaging filter, the filter coefficients are uniform and

sum to one.
That is, the kernels can be
 
1/9 1/9 1/9
h = 1/9 1/9 1/9 .
1/9 1/9 1/9

 
1/25 1/25 1/25 1/25 1/25
1/25 1/25 1/25 1/25 1/25
 
 /25
h=1 1/25 1/25 1/25 1/25 .

1/25 1/25 1/25 1/25 1/25
1/25 1/25 1/25 1/25 1/25

etc.

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Averaging filter

The operation of the averaging filter is

k k
1 X X
J(x, y ) = I (x − m, y − n)
k2
m=−k n=−k

What happens at J(0, 0)? Border effects.

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Gaussian filter
In a Gaussian filter, the filter coefficients are determined by
the 2-D Gaussian distribution
1 −(x 2 +y 2 )/2σ2
h(i, j) = e
2πσ 2

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Gaussian filter

A digital image contains discrete spatial coordinates.

Need to approximate the Gaussian distribution at discrete

spatial coordinates.

How large should the kernel be? Beyond µ ± 3σ most values

are negligible.

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra