EE 5623: Advanced Digital Image Processing: Instructor Yahoo Group: CUST - ADIP - SPR2016 Text Books and Notes

EE 5623: Advanced Digital Image Processing
Instructor Dr. Imtiaz Ahmad Taj (imtiaztaj@cust.edu.pk)

Yahoo Group: CUST_ADIP_SPR2016
https://groups.yahoo.com/neo/groups/cust_adip_spr2016
Text books and notes 1. R. C. Gonzalez and R. E. Woods,
“Digital Image Processing”, 3rd
edition, Pearson Education, Inc., 2008.
2. “Digital Image Processing using
MATLAB” R. C. Gonzalez , R. E.
Woods and S.L. Eddins Pearson
Education, Inc., 2004.
3. Class Slides
Additional books J. C. Russ, “The Image Processing Handbook”,
fourth edition, CRC Press, 2002.
2/16/2016 Capital University of Science and Technology Advanced Digital Image Processing EE5623 1
Course objectives
 To give a thorough understanding of advance tools,
algorithms and techniques used in image processing to
the students
 To enable students to be able solve an image processing
problem and to design systems on their own by giving
them hands on working experience on image processing
tools
 To introduce advanced topics in image processing and
invoke interest for doing research in this field
1
Pre-requisites for the course
1. Knowledge of probability and random variables,
Vectors and Matrices.
2. Working knowledge of Matlab
3. Signals and Systems, and DSP courses especially the
concepts of Convolution, FFT, filtering, etc.
Grading Policy: Breakup

 Midterm: ~20%
 Quizzes: ~15%
 Assignments ( written + programming) ~12%
 Project ~13%
 Final: ~40%
Grading breakup can change without any notice

during the semester in benefit of all the students
2
Assignments
 Please check the class folder every day, for notification of
assignments, projects and updated information.
 Assignments will have ~12% weight in the total marks.
 Assignments may be written assignments of programming
assignments.
 There will be a total of around 8 assignments.
 The deadline for the submission of assignment will be given with
the assignment.
 Assignments submitted after the deadline will not be accepted and
will carry ZERO MARKS.
 Cheated assignments will get NEGATIVE MARKS.
Project
 Projects will have ~15% weight in the total marks.
 Projects may be conducted individually or in groups of two
students.
 Different suggested topics for projects will be uploaded on the
class yahoo group within the first two weeks of the course. If you
want to do your own project take permission first.
 Reading material and other sources for every project to help the
students will also be given.
 Project topics should be selected and approved within the first five
weeks of the course.
 Project presentation date will be announced and projects will not
be accepted after the presentation date.
 Projects consisting of Downloaded codes or presentations will not
be accepted will carry ZERO MARKS.
3
Why do we process images?
 Facilitate picture storage and transmission
– Efficiently store an image in a digital camera
– Send an image through mobile phone
 Enhance and restore images
– Remove scratches from an old photo
– Improve visibility of tumor in a radiograph
 Extract information from images
– Measure water pollution from aerial images
– Measure the 3D distances and heights of objects from stereo images
 Prepare for display or printing
– Adjust image size
– Halftoning
 Biometrics based identification
Image Processing Examples

Photo restoration
Damaged Image Restored Image
4
Photo colorization
Original B/W Image colorized Image Original Image Colorized Image

Color photo enhancement
Original Images Enhanced Images
5
Image Enhancement
Initial image Final image
Performed steps: Gaussian blur, contrast enhancement, median filter

and sharpening. Processing made by Photoshop.
Example taken from microscopy.berkeley.edu/courses/dib

Restoration of image from Hubble Space Telescope
Faulty image of Saturn Recovered image
6
Extraction of settlement area from an aerial image

Earthquake analysis from space
Image shows the ground displacement of a typical area due to earthquake
7
 Medical Imaging: Computer Tomography (CT)
– Generating 3-D images from 2-D slices.
– CAD, CAM applications
– Industrial inspections

 Medical Imaging: Computer Aided Tomography (CAT)
8
 Medical Imaging: Ultrasound imaging

Medical imaging: Averaging MRI slices for knee image
9
Image compression
Original JPEG 27:1

Image compression
Original JPEG2000 27:1
10
Face detection

Face Tracking
11
Face Morphing

Fingerprint recognition
X X 
12
Personal Identification using Iris Recognition
1983 2000
National Geographics: “Afghan Girl”

Removing geometric distortions of camera
13
Applications of DIP
Categorization according to image sources
 Electromagnetic (EM) band Imaging
– Gamma ray images
– x-ray band images
– ultra-violet band images
– visual light and infra-red images
– Imaging based on micro-waves and radio waves
 Non-EM band Imaging
– Acoustic and ultrasonic images
– Electron Microscopy
– Computer-generated synthetic images
EM Spectrum
14
Applications of DIP: EM band imaging
 Gamma-ray imaging
– Nuclear medicine, astronomical observations.
 X-ray Imaging
– Medical diagnostics (CAT scans, x-ray scans), industry, astronomy.
 Ultra-violet imaging
– Fluorescence microscopy, astronomy,
 Visible & Infrared-band imaging (most widely used)

– Light microscopy, astronomy, remote sensing, industry, law enforcement,
military recognizance, etc.
 Micro-wave and radio band imagery

– Radar, Medicine (MRI), astronomy
Applications of DIP: Non-EM band imaging

 Acoustic imaging (hundreds of Hz)
– Geological exploration (oil exploration)
 Ultrasound imaging (millions of Hz)

– Industry and medicine especially in obstetrics, determine the health of the
fetal development
 Electron microscopic imaging

– Used to achieve magnification of 10,000x or more
• (Light microscopy is limited to around 1000x)
 Synthetic imaging
– 3D modeling or visualization systems for flight simulators, machine design,
special effects and animations,etc.
15
Classification of DIP and Computer Vision Processes
 Low-level process: (DIP)

– Primitive operations where inputs and outputs are images
Major functions: image pre-processing like noise reduction,
contrast enhancement, image sharpening, etc.
 Mid-level process (DIP and Computer Vision and Pattern

Recognition)
– Inputs are images, outputs are attributes (e.g., edges) major
functions: segmentation, description, classification /
recognition of objects
 High-level process (Computer Vision)

– make sense of an ensemble of recognized objects; perform the
cognitive functions normally associated with vision
Image Processing Steps

Physical world
Image acquisition
Imaging
Digitization, quantization and
compression
Enhancement and restoration Image Processing
Image segmentation Imaging Analysis
Feature selection/extraction (Computer Vision and
Pattern recognition)
Image representation
Image understanding
Image interpretation
(Computer Vision and
Pattern recognition)
Physical action
16
Image Processing Computer vision and PR
 Image acquisition by sensor Image Geometrical Rectification
 Image sampling and quantization  Camera geometry
Computer Vision
Feature Extraction
Image enhancement and restoration  Edge and Interest points detection
 Filtering in spatial domain or  Texture and shading
frequency domain  Shape from texture and shading
Feature Extraction Calculation on Multiple Views
 Edge detection  Multi-view geometry and Stereo imaging
 Interest points  Structure from motion
Colored image Processing Segmentation
 Pseudo coloring  Impose some order on group of pixels to
 Color segmentation separate them from each other
Multi-resolution analysis Template matching
Pattern Recognition
 Pyramids
 Wavelets Segmentation
 Other transformations Classification and Recognition
Image and video compression  Classification and interpretation of objects
 Image compression standards based on selected features
 Video compression standards  Recognize objects using probabilistic
techniques
Scope of DIP Course

 Digital image fundamentals and image acquisition (briefly)
 Image enhancement in spatial domain
– pixel operations
– histogram processing
– Filtering
 Image enhancement in frequency domain

– Transformation and reverse transformation
– Frequency domain filters
– Homomorphic filtering
 Image sampling
 Image restoration
– Noise reduction techniques
– Geometric transformations
17
Scope of DIP Course (continued)
 Color image processing
– Color models
– Pseudocolor image processing
– Color transformations and color segmentation
 Wavelets and multi-resolution processing

– Multi-resolution expansion
– Wavelet transforms, etc.
 Image compression
– Image compression models
– Error free compression
– Lossy compression, etc
Scope of DIP Course (continued)

 Morphological image processing
– Dilation and erosion
– Opening closing
– Hit or miss transformation
– Basic morphological algorithms
18
Image Acquisition
Sampling and Quantization quantization
sampling sampling
19
Sampling and Quantization
Sampling: Digitization of the spatial coordinates (x,y)
Quantization: Digitization in amplitude (also called gray-

level quantization)
8 bit quantization: 28 = 256 gray levels (0: black, 255: white)
Binary (1 bit quantization): 2 gray levels (0: black, 1: white)
Commonly used number of samples (resolution)
Digital still cameras: 640x480, 1024x1024, 4064 x 2704 so on
Digital video cameras: 640x480 at 30 frames/second
1920x1080 at 60 f/s (HDTV)
Sampling and Quantization
An M x N digital image is expressed as

Columns
 f (0,0) f (0,1) . . . f (0, N  1) 
 f (1,0) f (1,1) . . . f (1, N  1) 
Rows
 
 . . . . . . 
 
 . . . . . . 
 . . . . . . 
 
 f ( M  1,0) f ( M  1,1) . . . f ( M  1, N  1)
N : No of Columns
M : No of Rows
20
Digital Images
Digital images are 2D arrays (matrices) of numbers:
Sampling
21
Sampling
Effect of Sampling and Quantization
250 x 210 samples 125 x 105 samples 50 x 42 samples 25 x 21 samples

256 gray levels
16 gray levels 8 gray levels 4 gray levels Binary image

22
RGB (color) Images
Red + Blue + Green
Red Green Blue

Image acquisition
Single imaging sensor
Line sensor
Array sensor
23
Image acquisition
Image acquisition
Image acquisition through

linear sensor strip
Image acquisition through

circular sensor strip
24
Digital Camera Technologies
 CCD (Charge Coupled Device)
– Capacitive device
– Proper mechanism for charge
transfer
 CMOS (Complementary magnetic
oxide)
– Fabricated in standard
semiconductor production line
– Lower power consumption A CCD:
2–5 W, CMOS: 20–50 mW
 Foveon X3
– a chip of transparent quartz
containing 3 layers of CMOS
CCD Array Cameras

Consists of sensor elements/ photo detectors (active devices) and
charge storage devices also called charge buckets
Every element in the array is linked (charge coupled) to other
element.
Charges are transferred serially out of the array through shifting
charges from one element to the other.
25
CCD Array Cameras
Frame Transfer
Architecture

CCD Array Cameras
Interline Transfer
Architecture
26
CMOS Array Cameras

Standard semiconductor
production line
Active pixel architecture
Photo-detector and amplifier
are both fabricated inside
each pixel.
Digital camera technologies comparison

CCD (Charge Coupled Device) CMOS (Complementary Metal
–Specialized fabrication Oxide Semiconductor)
techniques are used so –Cheaper technology
expensive technology –Smaller size
–Larger size –Low power consumption
–Higher power consumption –Readout for selective area of an
because of the capacitive image is possible
architecture
–Amplifier and additional
–Always have to read out the circuitry can be fabricated
whole image inside each pixel.
–Resolution is limited by –Higher resolution possible
sensor elements size
–Stronger noise due to higher
–Less on-chip circuitry so dark currents because of more
lesser dark currents and on-chip circuitry
noiseCapital University of Science and Technology
2/16/2016 Advanced Digital Image Processing EE5623 54
27
Acquisition of color images
Single sensor assembly
For still scenes
Three sensors
with prisms
Sensor arrays
a. Stripe filter pattern
b. Bayers filter pattern
Acquisition of color images

 Fabrication of CMOS colored sensors
28
Foveon X3 imager
 A recent development in
photo-sensing technology
 This sensor uses 3 layers of
CMOS imagers.
 Each layer absorbs different
colours of light at different
depths.
 Single shot camera for three
colors
 Foveon has failed to make a
serious impact on the
market.
Scanning Schemes
Interlaced scanning
(used in TV)
 Read/display all
even-numbered lines
(even field, half-size)
 Restart
 Read/display all odd-
A typical Interlaced Scanning scheme
numbered lines (odd
field, half-size)
 Stitch the even and
odd fields together
and form a single,
full-size frame
 Output
2/16/2016 theUniversity
Capital full-sizeof Science and Technology Advanced Digital Image Processing EE5623 58
frame
29
Interlaced scanning
When motion is present the interlaced scanning produces blurring
in the image
Scanning Schemes
Progressive Scanning
 Immediately transfer an entire frame at once from
the image sensor without performing any line-
interlacing.
 Suitable for fast motion detection applications
 Incompatible with standard television systems.
 Popular in digital cameras (computer applications)
30
Various commercial sensor sizes
Aspect Width Height

"Name"
Ratio (mm) (mm)
1/3.6" 4:3 4.0 3.0
1/3.2" 4:3 4.5 3.4
1/3" 4:3 4.8 3.6
1/2.7" 4:3 5.3 4.0
1/2" 4:3 6.4 4.8
1/1.8" 4:3 7.2 5.3
2/3" 4:3 8.8 6.6

Relative size of various digital camera sensors
1" 4:3 12.8 9.6
4/3" 4:3 18.0 13.5
EOS 10D 3:2 22.0 15.0
31

EE 5623: Advanced Digital Image Processing: Instructor Yahoo Group: CUST - ADIP - SPR2016 Text Books and Notes

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

EE 5623: Advanced Digital Image Processing: Instructor Yahoo Group: CUST - ADIP - SPR2016 Text Books and Notes

Uploaded by

Copyright:

Available Formats

EE 5623: Advanced Digital Image Processing

Instructor Dr. Imtiaz Ahmad Taj (imtiaztaj@cust.edu.pk)

Grading Policy: Breakup

Grading breakup can change without any notice

Image Processing Examples

Damaged Image Restored Image

Original B/W Image colorized Image Original Image Colorized Image

Image Processing Examples

Original Images Enhanced Images

Performed steps: Gaussian blur, contrast enhancement, median filter

Example taken from microscopy.berkeley.edu/courses/dib

Image Processing Examples

Restoration of image from Hubble Space Telescope

Faulty image of Saturn Recovered image

Image Processing Examples

Image shows the ground displacement of a typical area due to earthquake

Image Processing Examples

Image Processing Examples

Original JPEG 27:1

Image Processing Examples

Original JPEG2000 27:1

Image Processing Examples

Image Processing Examples

National Geographics: “Afghan Girl”

Removing geometric distortions of camera

 Visible & Infrared-band imaging (most widely used)

 Micro-wave and radio band imagery

Applications of DIP: Non-EM band imaging

 Ultrasound imaging (millions of Hz)

 Electron microscopic imaging

 Low-level process: (DIP)

 Mid-level process (DIP and Computer Vision and Pattern

 High-level process (Computer Vision)

Image Processing Steps

Scope of DIP Course

 Image enhancement in frequency domain

 Wavelets and multi-resolution processing

Scope of DIP Course (continued)

Sampling and Quantization quantization

Quantization: Digitization in amplitude (also called gray-

Sampling and Quantization

An M x N digital image is expressed as

Digital images are 2D arrays (matrices) of numbers:

Effect of Sampling and Quantization

250 x 210 samples 125 x 105 samples 50 x 42 samples 25 x 21 samples

16 gray levels 8 gray levels 4 gray levels Binary image

Red + Blue + Green

Red Green Blue

Single imaging sensor

Image acquisition through

Image acquisition through

Digital Camera Technologies

CCD Array Cameras

Digital Camera Technologies

CMOS Array Cameras

Digital camera technologies comparison

Acquisition of color images

Aspect Width Height

1/3.6" 4:3 4.0 3.0

1/3.2" 4:3 4.5 3.4

1/3" 4:3 4.8 3.6

1/2.7" 4:3 5.3 4.0