Department of Mechatronics

Image Acquisition and Pre-processing

MACHINE VISION AND IMAGE PROCESSING
(MTE 4075)

Year/ Semester : IVth / VII th

Name of Faculty:
Dr. Umesh Sahu
(Assistant Professor)
Manipal Institute of Technology, Manipal, India
−1−
Syllabus
Image acquisition and pre-processing: [15]
Vision system components, Image acquisition and analysis, Image
formation - Sampling & Quantization, simple operations on image,
Image enhancement Spatial Domain and frequency domain methods,
Image noise, image restoration, morphological operations,
Segmentation, image analysis, representation of regions,
Feature extraction, fundamentals of color image processing,
conversion of color spaces.

Manipal Institute of Technology, Manipal, India

−2−
Content:
▪Feature Extraction
–Introduction
–Representation and Description
–Feature Extraction
•Region Feature
•Line Feature
•Point Feature

Manipal Institute of Technology, Manipal, India

−3−
Image Features
▪ Motivation
– One of the major concern of image processing is object recognition
– Our task is to describe the region based on the selected representation

Image acquisition → Digital Image

Pre-processing → Enhanced Image

Segmentation → Desirable region

Representation and description → Features

Manipal Institute of Technology, Manipal, India

−4−
Representation and Description
• After segmenting the resulting segmented pixels usually is represented and
described in a form for further computer processing.

• Representing the regions involves two choices:

• By external characteristics (boundary)
• By internal characteristics (the pixels comprising the region)

• Describe the region based on the selected representation:

Example: A region may be represented by its boundary, and the boundary
described by the features (such as length, orientation etc..)

Manipal Institute of Technology, Manipal, India

−5−
Desirable Properties of Descriptors
▪ They should define a complete set
- Two objects must have the same descriptors if and only if they have the same
shape.

▪ They should be invariant to Rotation, Scaling and Translation (RST).

▪ A descriptor should only contain information about what makes an object

unique, or different from the other objects.

▪ They should be robust

-Work well against Noise and Distortion

▪ They should have low computational complexity

Manipal Institute of Technology, Manipal, India
−6−
 Feature Extraction
▪ The common goal of feature extraction and representation
techniques is to convert the segmented objects into
representations that better describe their main features and
attributes.

Corke, Peter I., and Oussama Khatib. Robotics, Vision and Control Fundamental algorithms in MATLAB:
First/Second Edition, Springer. (Chapter-13)

Manipal Institute of Technology, Manipal, India

−7−
About:
Robotics and Machine Vision Toolbox

https://petercorke.com/rvc/home/

Manipal Institute of Technology, Manipal, India

−8−
 Feature Extraction
• Feature extraction operates on an image and returns one or more image features.

• Features are typically:

• Scalars (for example area or aspect ratio).
• Short vectors (for example the coordinate of an object or the parameters of a line).

• Image feature extraction is a necessary first step in using image data to control a robot.

• It is an information concentration step:

• Reduces the data rate from 106−108 bytes/second at the output of a camera to
something of the order of tens of features per frame.
• That can be used as input to a robot’s control system.
Manipal Institute of Technology, Manipal, India
−9−
 Feature Extraction
▪ Classes of feature:
–Regions
–Lines
–Interest points

Manipal Institute of Technology, Manipal, India

− 10 −
 Region Feature:
▪ Image segmentation is considered
as three subproblems:
–Classification
• Grey-Level Classification
• Color Classification
–Representation
–Description MATLAB Demo
in Robotics and
MV Toolbox

feat_ext1.m
Manipal Institute of Technology, Manipal, India
− 11 −
 Region Feature:
▪ Image segmentation is considered as three subproblems:
–Classification
• Grey-Level Classification (ithresh(), otsu(), niblack(), imser())
• Color Classification (colorkmeans())

–Representation (ilabel())
–Description
• Bounding Boxes
• Moments
• Blob Features
• Shape from Moments
• Shape from Perimeter
• Character Recognition
Manipal Institute of Technology, Manipal, India
− 12 −
 Region Feature: Description
• In Representation we have find connected components in the image and
how to isolate particular components.
• However this representation of the component is still just an image with
logical pixel values rather than a concise description of its size, position
and shape.

Bounding Boxes : MATLAB Demo

in Robotics and
• The simplest representation of size and shape is the bounding MV Toolbox
box.
• the smallest rectangle with sides parallel to the u- and v-axes
that encloses the region.
feat_ext_2.m
Manipal Institute of Technology, Manipal, India
− 13 −
Moment Feature
▪ Moments are a rich and computationally cheap class of image features which
can describe region size and location as well as shape.
▪ The moment of an image I is a scalar
MATLAB Demo
in Robotics and
MV Toolbox

▪ where (p +q) is the order of the moment. The zeroth moment p =q = 0 is

▪ For a binary image where the background pixels are zero this is simply the
number of non-zero (white) pixels – the area of the region.
Moments are calculated using the Toolbox function mpq
Manipal Institute of Technology, Manipal, India
− 14 −
Moment Feature

▪ Moments can be given a physical interpretation by regarding the image function

as a mass distribution.
▪ Example: Consider the region as being made out of thin metal plate where
each pixel has one unit of area and one unit of mass.
– The total mass of the region is m00
– The centre of mass or centroid of the region is

▪ where m10 and m01 are the first-order moments.

Manipal Institute of Technology, Manipal, India

− 15 −
Central moments

▪The central moments μpq are computed with respect to the

centroid

▪μpq are Invariant to the position of the region.

▪They are related to the moments mpq by

Central Moments are calculated using the Toolbox function upq

Manipal Institute of Technology, Manipal, India
− 16 −
 Central Moments

▪Using the thin plate analogy again, the inertia matrix of the
region (about axes parallel to the u- and v-axes and
intersecting at the centroid of the region.) is

μ20, μ02 : The moments of inertia

μ11 : The product of inertia.

Central Moments are calculated using the Toolbox function upq

Manipal Institute of Technology, Manipal, India
− 17 −
Hu’s Invariant Moments

• Combinations of moments are invariant to translation, scale, and

rotation
• Can be used for recognition
1 = η20 + η02
2 = (η20 − η02)2 +(2η11)2
3 = (η30 − 3η12)2 + (3η21 − η03)2
4 = (η30 + η12)2 + (η21 + η03)2
5 = (η30 − 3η12)(η30 + η12)[(η30 + η12)2 − 3(η21 + η03)2] + (3η21 − η03)(η21 + η03)[3(η30 + η12)2 − (η21 +
η03)2]

6 = (η20 − η02)[(η30 + η12)2 − (η21 + η03)2] + 4η11(η30 + η12)(η21 + η03)

7 = (3η21 − η03)(η30 + η12)[(η30 + η12)2 − 3(η21 + η03)2] − (η30 − 3η12)(η21 + η03)[3(η30 + η12)2 − (η21 +
η03)2].

Manipal Institute of Technology, Manipal, India

− 18 −
Normalized Central Moments

• Normalized central moments (divide by area raised to a power)

 p,q
 p,q =
(0,0 )

p+q
where  = + 1, for p + q = 2,3,
2

Manipal Institute of Technology, Manipal, India

− 19 −
Region features and their invariance to camera motion: translation,
rotation about the object’s centroid and scale factor

Manipal Institute of Technology, Manipal, India

− 20 −
 MATLAB Demo
in Robotics and
MV Toolbox

feat_ext_2.m
Manipal Institute of Technology, Manipal, India
− 21 −
 Region Feature:
▪ Image segmentation is considered as three subproblems:
–Classification
–Representation
–Description
• Bounding Boxes
• Moments (mpq(), upq()) MATLAB Demo
in Robotics and
• Blob Features (imoments(), iblobs()) MV Toolbox

• Shape from Moments (humoments())

• Shape from Perimeter (self Learning topic)
• Character Recognition (ocr())

feat_ext_2.m
Manipal Institute of Technology, Manipal, India
− 26 −
 Line Feature:
• Lines are distinct visual features that are particularly common in man-
made environments
– for example the edges of roads, buildings and doorways.

• In segmentation we have discussed how image intensity gradients can be

used to find edges within an image,

• Here will be concerned with fitting line segments to such edges.

MATLAB Demo
Corke, Peter I., and Oussama Khatib. Robotics, Vision and Control Fundamental in Robotics and
MV Toolbox
algorithms in MATLAB: First/Second Edition, Springer. (Chapter-13)

feat_ext_3.m
Manipal Institute of Technology, Manipal, India
− 27 −
 Point Feature:
❑ These are visually distinct points in the image that are known as
interest points, salient points, key points or corner points.

❑ Classical Corner Detectors

1. Shi-Tomasi detector
2. The Harris detector (icorner()) Self Learning Topic
3. Noble detector

Corke, Peter I., and Oussama Khatib. Robotics, Vision and Control Fundamental algorithms in MATLAB:
Second Edition, Springer. (Page:443)

Manipal Institute of Technology, Manipal, India

− 28 −
 Point Feature:
❑ Scale-Space Corner Detectors:
❑ The Harris corner detector works very well in practice, but
responds poorly to changes in scale.
❑ iscalespace()
❑ iscalemax()

❑ Scale-Space Point Feature: isurf()

MATLAB Demo
❑ The Scale-Invariant Feature Transform (SIFT) is based on the maxima in Robotics and
in a difference of Gaussian sequence. MV Toolbox

❑ The Speeded Up Robust Feature (SURF) is based on the maxima in

an approximate Hessian of Gaussian sequence.

feat_ext4.m
Manipal Institute of Technology, Manipal, India
− 29 −
 Point Feature: The Harris detector
icorner()

Scale-Space Corner Detectors

iscalespace()
Scale-Space
Point Detectors
isurf()

Manipal Institute of Technology, Manipal, India

− 30 −
Summary
▪We have looked at:
▪Feature Extraction
–Introduction
–Representation and Description
–Feature Extraction
•Region Feature
•Line Feature
•Point Feature

Manipal Institute of Technology, Manipal, India

− 31 −
Thank you
for your attention

Manipal Institute of Technology, Manipal, India

− 32 −