Eece253 01 Intro

EECE/CS 253 Image Processing
Lecture Notes: Introduction and Overview
Richard Alan Peters II Department of Electrical Engineering and Computer Science Fall Semester 2011
. 1. %. {. . {. . . . . .
This work is licensed under the Creative Commons Attribution-Noncommercial 2.5 License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc/2.5/ or send a letter to Creative Commons, 543 Howard Street, 5th Floor, San Francisco, California, 94105, USA.
Introduction and Overview

This presentation is an overview of some of the ideas and techniques to be covered during the course.
25 August 2011
1999-2011 by Richard Alan Peters II
Topics
1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11.
Image formation Point processing and equalization Color correction The Fourier transform Convolution Image sampling, warping, and stitching Spatial filtering Noise reduction Mathematical morphology High dynamic range imaging Image compression
1999-2011 by Richard Alan Peters II 3
25 August 2011
Wallace and Gromit

Wallace
reads Electronics for Dogs
Gromit
likes cheese
Wallace and Gromit will be subjects of some of the imagery in this introduction.
http://www.aardman.com/wallaceandgromit/index.shtml
25 August 2011 1999-2011 by Richard Alan Peters II 4
Image Formation
25 August 2011
Image Formation
25 August 2011
Image Formation
projection through lens
image of object
25 August 2011
Image Formation
projection onto discrete sensor array.
digital camera
25 August 2011
Image Formation
sensors register average color.
sampled image
25 August 2011
Image Formation
continuous colors, discrete locations.
discrete realvalued image

10
25 August 2011
Digital Image Formation: Quantization

discrete color output
continuous colors mapped to a finite, discrete set of colors.
continuous color input

Sampling and Quantization

pixel grid
real image
sampled
quantized
sampled & quantized
25 August 2011
12
Digital Image
a grid of squares, each of which contains a single color
Color images have 3 values per pixel; monochrome images have 1 value per pixel.
each square is called a pixel (for picture element)
25 August 2011
13
Color Images

Are constructed from three intensity maps. Each intensity map is projected through a color filter (e.g., red, green, or blue, or cyan, magenta, or yellow) to create a monochrome image. The intensity maps are overlaid to create a color image. Each pixel in a color image is a three element vector.
Color Images On a CRT
25 August 2011
15
Point Processing
- gamma
- brightness
original
+ brightness
+ gamma
histogram mod
25 August 2011
- contrast
original
+ contrast
histogram EQ
16
Color Processing
requires some knowledge of how we see colors
25 August 2011
17
Eyes Light Sensors

cone density near fovea
#(blue) << #(red) < #(green)

Color Sensing / Color Perception

These are approximations of the responses to the visible spectrum of the red, green, and blue receptors of a typical human eye.
25 August 2011
19

These are approximations of the responses to the visible spectrum of the red, green, and blue receptors of a typical human eye.
The simultaneous red + blue response causes us to perceive a continuous range of hues on a circle. No hue is greater than or less than any other hue.
25 August 2011
20

luminance 25 August 2011 hue saturation
The eye has 3 types of photoreceptors: sensitive to red, green, or blue light.
The brain transforms RGB into separate brightness and color channels (e.g., LHS).
brain photo receptors
21
Color Perception
16 pixelization of:
luminance and chrominance (hue+saturation) are perceived with different resolutions, as are red, green and blue.
all bands
luminance
chrominance
red
25 August 2011
green
blue
22
Color Perception
16 pixelization of:
all bands
luminance
chrominance
red
25 August 2011
green
blue
23
Color Balance and Saturation

Uniform changes in color components result in change of tint. E.g., if all G pixel values are multiplied by > 1 then the image takes a green cast.
25 August 2011
24
Color Transformations
222 222 218 17 121 171 17 122 114 222 222 185
240 230 166 240 171 160
236 227 106 240 171 103
Image aging: a transformation,

17 122 114 17 121 171 222 222 185 222 222 218 240 171 103
, that mapped:
240 171 160 236 227 106 240 230 166
25 August 2011
25
The 2D Fourier Transform of a Digital Image

Let I(r,c) be a single-band (intensity) digital image with R rows and C columns. Then, I(r,c) has Fourier representation
R 1 C 1
I r, c
u 0 v 0
I u ,v e
i2
ur R
vc C
where
these complex exponentials are 2D sinusoids.
I u,v
1 RC
R 1C 1
i2
I ( r ,c ) e
r 0c 0
ur R
vc C
are the R x C Fourier coefficients.

2D Sinusoids:
... are plane waves with grayscale amplitudes, periods in terms of lengths, ...
I r, c
A 2 cos 2
c cos C
r sin R
orientation
A = phase shift
2D Sinusoids:
c
... specific orientations, and phase shifts.
25 August 2011
28
The Value of a Fourier Coefficient

is a complex number with a real part and an imaginary part. If you represent that number as a magnitude, A, and a phase, , ..these represent the amplitude and offset of the sinusoid with frequency and direction .
The Sinusoid from the Fourier Coeff. at (u,v)
Here is the same coefficient plotted as magnitude, A, and a phase, , and displayed in the space domain as a sinusoid.
The Fourier Transform of an Image
magnitude
I
25 August 2011
phase
[F{I}]
31
|F{I}|
Continuous Fourier Transform

I u, v e
i 2 ( uc vr )
I r, c
dudv
I u, v
I r, c e
i 2 ( uc vr )
dcdr
The continuous Fourier transform assumes a continuous image exists in a finite region of an infinite plane.
The BoingBoing Bloggers
25 August 2011
32
Discrete Fourier Transform

The discrete Fourier transform assumes a digital image exists on a closed surface, a torus.
R 1C 1
I (r,c)
v 0u 0
i2
u,v e
uc C
vr R
I
The BoingBoing Bloggers
R 1C 1
u, v
r 0c 0
I r ,c e
i2
cu C
rv R
25 August 2011
33
Convolution
r 16, c 16 r 16, c 16
Sums of shifted and weighted copies of images or Fourier transforms.
r 0, c 0
r 16, c 16
r 16, c 16
Sum times 1/5

Convolution Property of the Fourier Transform

Let functions f ( r, c ) and g ( r, c ) have Fourier Transforms F (u, v ) and G (u, v ). Then,
F{f
M oreover,
g}
F G.
g } F G. represents convolutio n represents pointwise multiplica tion Then, a spatial convolution can be computed by f g
F{f
The Fourier Transform of a product equals the convolution of the Fourier Transforms. Similarly, the Fourier Transform of a convolution is the product of the Fourier Transforms
-1
F G.
25 August 2011
35
Sampling, Aliasing, & Frequency Convolution
aliasing (the jaggies)

25 August 2011
no aliasing (smooth lines)

36
Sampling, Aliasing, & Frequency Convolution

(a) (b)
(a) (b) (c) (d)
aliased power spectrum unaliased power spectrum
(c) 1999-2011 by Richard Alan Peters II
(d) 37
25 August 2011
Resampling
nearest neighbor nearest neighbor
16
(resizing)
bicubic interpolation
25 August 2011 1999-2011 by Richard Alan Peters II
bicubic interpolation
38
Rotation
25 August 2011
and motion blur

39
Image Warping
25 August 2011
40
Bruno Postle http://hugin.sourceforge.net/tutorials/two-photos/en.shtml
Panorama via Overlay

Originals
Merged*
25 August 2011
*not
so good.
41
Bruno Postle http://hugin.sourceforge.net/tutorials/two-photos/en.shtml
Panorama via Stitching

Originals
Merged*
25 August 2011
*much
better.
42
Frequency Domain (FD) Filtering
Image size: 512x512 SD filter sigma = 8
Original Image
Power Spectrum
Gaussian LPF in FD
25 August 2011
43
FD Filtering: Lowpass
Image size: 512x512 SD filter sigma = 8
Filtered Image
Filtered Power Spectrum
Original Image
25 August 2011
44
FD Filtering: Highpass
Image size: 512x512 FD notch sigma = 8
Filtered Image
Original Image
25 August 2011
45
FD Filtering: Highpass
signed image with 0 at middle gray
Image size: 512x512 FD notch sigma = 8
Filtered Image
Original Image
25 August 2011
46
Spatial Filtering
blurred
original
sharpened
25 August 2011
47
Spatial Filtering
bandpass filter
original
unsharp masking
25 August 2011
48
Spatial Filtering
signed image with 0 at middle gray
bandpass filter
original
unsharp masking
25 August 2011
49
Motion Blur
regional vertical
original zoom
25 August 2011
rotational
50
Noise Reduction
blurred image
25 August 2011
color noise
color-only blur
51
Noise Reduction
blurred image
25 August 2011
color noise
5x5 Wiener filter

52
Noise Reduction
periodic noise
25 August 2011
original
frequency tuned filter

53
Shot Noise or Salt & Pepper Noise
+ shot noise
25 August 2011
s&p noise
- shot noise
54
Nonlinear Filters: the Median
original
25 August 2011
s&p noise
median filter
55
Nonlinear Filters: Min and Maxmin
+ shot noise
25 August 2011
min filter
maxmin filter
56
Nonlinear Filters: Max and Minmax
- shot noise
25 August 2011
max filter
minmax
57
Nonlinear Processing: Binary Morphology
L shaped SE O marks origin

25 August 2011
Foreground: white pixels Background: black pixels

Cross-hatched pixels are indeterminate.
58
Nonlinear Processing: Binary Reconstruction
Used after opening to grow back pieces of the original image that are connected to the opening. Permits the removal of small regions that are disjoint from larger objects without distorting the small features of the large objects.
original
25 August 2011
opened
reconstructed
59
Nonlinear Processing: Grayscale Morphology
L shaped SE O marks origin

25 August 2011
Foreground: white pixels Background: black pixels

Cross-hatched pixels are indeterminate.
60
Grayscale Morphology: Opening
opening: erosion then dilation
opened & original
25 August 2011
61
Grayscale Morphology: Opening
erosion & opening
erosion & opening & original
25 August 2011
62
Nonlinear Processing: Grayscale Reconstruction

original reconstructed opening
25 August 2011
63
Forensic Analysis of Photographs

Which came first?
Photographs by Robert Fenton of a battlefield in the Crimean war taken on 23 April 1855. From Morris, Errol, Which Came First, the Chicken or the Egg?, Parts 1-3, New York Times, Zoom Editorial Section, 25 Sept. 2007 (pt.1), 7 Oct. 2007 (pt.2), 30 Oct. 2007 (pt.3).
Which came first?
25 August 2011
65
Which came first?
25 August 2011
66
High Dynamic Range (HDR) Imaging
Bartlomiej Okonek http://www.easyhdr.com/examples.php
under exposed
default exposure
over exposed
combined
Image Compression
Original image is 5244w x 4716h @ 1200 ppi: 127MBytes
Yoyogi Park, Tokyo, October 1999. Photo by Alan Peters.

Image Compression: JPEG

JPEG quality level File size in bytes
25 August 2011
72
Image Compression: JPEG

JPEG quality level File size in bytes
25 August 2011
73
Image Compositing

Combine parts from separate images to form a new image. Its difficult to do well. Requires relative positions, orientations, and scales to be correct. Lighting of objects must be consistent within the separate images. Brightness, contrast, color balance, and saturation must match. Noise color, amplitude, and patterns must be seamless.
25 August 2011
74
Image Compositing Example
Prof. Peters in his home office. Needs a better shirt.

This shirt demands a monogram.

He needs some more color.

Nice. Now for the way hed wear his hair if he had any.
He cant stay in the office like this.

Wheres a hepcat Daddy-O like this belong?


Collar this jive, Jackson. Like crazy, Man !
In the studio!

Eece253 01 Intro

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Eece253 01 Intro

Uploaded by

Copyright:

Available Formats

EECE/CS 253 Image Processing

Lecture Notes: Introduction and Overview

Introduction and Overview

1999-2011 by Richard Alan Peters II

Wallace and Gromit

1999-2011 by Richard Alan Peters II

1999-2011 by Richard Alan Peters II

projection through lens

1999-2011 by Richard Alan Peters II

projection onto discrete sensor array.

1999-2011 by Richard Alan Peters II

sensors register average color.

1999-2011 by Richard Alan Peters II

continuous colors, discrete locations.

discrete realvalued image

1999-2011 by Richard Alan Peters II

Digital Image Formation: Quantization

continuous colors mapped to a finite, discrete set of colors.

continuous color input

Sampling and Quantization

sampled & quantized

1999-2011 by Richard Alan Peters II

each square is called a pixel (for picture element)

1999-2011 by Richard Alan Peters II

Color Images On a CRT

1999-2011 by Richard Alan Peters II

1999-2011 by Richard Alan Peters II

requires some knowledge of how we see colors

1999-2011 by Richard Alan Peters II

Eyes Light Sensors

#(blue) << #(red) < #(green)

Color Sensing / Color Perception

1999-2011 by Richard Alan Peters II

Color Sensing / Color Perception

1999-2011 by Richard Alan Peters II

Color Sensing / Color Perception

1999-2011 by Richard Alan Peters II

Color Balance and Saturation

1999-2011 by Richard Alan Peters II

240 230 166 240 171 160

236 227 106 240 171 103

Image aging: a transformation,

1999-2011 by Richard Alan Peters II

The 2D Fourier Transform of a Digital Image

these complex exponentials are 2D sinusoids.

are the R x C Fourier coefficients.

... specific orientations, and phase shifts.

1999-2011 by Richard Alan Peters II

The Value of a Fourier Coefficient

The Sinusoid from the Fourier Coeff. at (u,v)

The Fourier Transform of an Image

Continuous Fourier Transform

1999-2011 by Richard Alan Peters II

Discrete Fourier Transform

1999-2011 by Richard Alan Peters II

Sums of shifted and weighted copies of images or Fourier transforms.

Sum times 1/5

Convolution Property of the Fourier Transform

1999-2011 by Richard Alan Peters II

Sampling, Aliasing, & Frequency Convolution

aliasing (the jaggies)

no aliasing (smooth lines)

1999-2011 by Richard Alan Peters II

Sampling, Aliasing, & Frequency Convolution