ME4821 Digital Image

Processing
Image Representation
& Description

Dr. Faraz Junejo

 Introduction
 This lecture discusses methods and techniques for
representing and describing an image and its objects
or regions of interest.

 Techniques presented in this lecture assume that an

image has undergone segmentation.

 The common goal of feature extraction and

representation techniques is to convert the
segmented objects into representations that better
describe their main features and attributes.
 Overview
In this lecture we will learn about how a
segmented image can be represented using its
internal and external features. We will look at:
What is Representation?
What is Description?
 Representation techniques
 Boundary Descriptors
 Regional Descriptors
Representation and Description
 Representation and Description (contd.)
After segmentation, the image needs to be
described and interpreted

Representation: an object may be represented by

its boundary.

Description: the object boundary may be described

by its length, orientation, or number of concavities.
 Representation: Overview
 Representing segmented region in two ways:

– in terms of its external characteristics (its boundary

pixels) i.e. focus on shape characteristics

– in terms of its internal characteristics (its internal

pixels) i.e. focus on regional properties, e.g., color,
texture

 Sometimes, we may need to use both ways

 Description: Overview
Description describes the region based on the
chosen representation
For example,
– representation  boundary
– description  length of the boundary,
orientation of the straight line joining its
extreme points, and the number of
concavities in the boundary.
 Sensitivity
Feature selected as descriptors should
be as insensitive as possible to variations
in
– size
– translation
– Rotation
i.e. the features should be scale, translation
and rotation invariant
 Chain codes
 Represent a boundary by:
- Using a logically connected sequence of straight-line segments,
- The line segments specify length and direction

 Using chain codes to represent the boundary results in a

significant reduction in the amount of data required to store the
boundary.

 Typically, this representation is based on 4 or 8-connectivity of

the segments.

 The direction of each segment is coded by using a numbering

scheme, as illustrated in following slides.
 Pixel Connectivity
 Connectivity: to trace contours, define object
boundaries, once segmentation has taken place.
 In order for two pixels to be connected, they must be
“neighbors” sharing a common property, i.e. satisfy
some similarity criterion.
 In a binary image with pixel values “0” and “1”, two
neighboring pixels are said to be connected if they
have the same value.
 Two types of connectivity is generally used:
- 4 connectivity
- 8 connectivity
 Relationship between pixels
Neighbors of a pixel
4-neighbors (N,S,W,E pixels) == N4(p).
A pixel p at coordinates (x,y) has four
horizontal and vertical neighbors:
(x+1,y), (x-1, y), (x,y+1), (x, y-1)
 8 connectivity
 You can add the four diagonal neighbors to give the
8-neighbor set.
8-neighbors: include diagonal pixels as well ==
N8(p).
 Pixel Connectivity:Summary
 Connectivity is used to trace contours, to define object
boundaries, and for segmentation.
 In order for two pixels to be connected, they must be
 “neighbors” sharing a common property—satisfy some i.e.
similarity criterion.
 In a binary image with pixel values “0” and “1”, two neighboring
pixels are said to be connected if they have the same value.
 4-connectivity: Two pixels p and q are 4-adjacent if q is in
the set N4(p).
 8-connectivity: q is in the set N8(p).
Binary Image
Digital boundary
 Chain Code: Procedure
 The chain code of a region can start at any pixel on the
boundary, normally it is taken as left most, top most pixel
though.

 It proceeds by finding the next adjacent pixel on the boundary

in a counter-clockwise direction, saving the direction (0-4) or
(0-7) in an output buffer, and then continuing the process
from the new pixel.

 When we arrive at the starting pixel again, the chain code is

complete. The output buffer contains a set of direction values
which comprise the chain code itself, and from which the
original set of pixels can be recreated starting at any pixel
position in an image.
 The simplest chain code
mechanism i.e. 4-directional, also
known as crack code, consists of
assigning a number to the direction
followed by a bug tracking
algorithm as follows:
right (0),up (1), left (2), and down (3).

Assuming that the total number of boundary points is p (the

perimeter of the contour), the array C (of size p), where C(p) = 0,
1, 2, 3, contains the chain code of the boundary.

A modified version of the basic chain code, known as the

Freeman code, uses eight directions instead of four.
 4-directional chain code: Example

Direction numbers for 4-

Directional chain code

4-directional chain code: Digital Boundary

0033333323221211101101
It should be noted generally, the starting point is chosen
at the top-most, left-most point of the boundary.
Exercise
 Exercise: solution

• 0, 0, 0, 3, 0, 0, 3, 3, 3, 3, 3, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 1
 8-directional chain code: Example

Direction numbers for 8-

Directionl chain code
8-directional chain code: Digital Boundary
076666553321212
It should be noted generally, the starting point is chosen at
the top-most, left-most point of the boundary.
(a) original contour;
(b) subsampled version of the
contour;
(c) chain code representation;
(d) Freeman code representation.
 Example
 Chain code for the region in this
figure is 446567001232.

 The chain code describes the

shape of the region
unambiguously, although its
position is completely unknown.

 Shape related measures such as

perimeter and area (number of
pixels in the region) can be
determined directly from the
chain code description alone.
 Chain codes (contd.)
Chain codes may be unacceptable because:
– the resulting chain of codes tends to be
quite long
– any small disturbances along the boundary
due to noise or imperfect segmentation
cause changes in the code that may not be
related to the shape of the boundary
 Chain codes (contd.)
Solve the problems by
– resample the boundary by selecting a
larger grid spacing
– however, different grid can generate
different chain codes
starting point is arbitrary
 Two Problems with the Chain Code
Chain code representation is conceptually
appealing, yet has the following two problems
– Dependent on the starting point
– Dependent on the orientation

To use boundary representation in object

recognition, we need to achieve invariance to
starting point and orientation
– Normalized codes
– Differential codes
 Normalized Chain Code (NCC)
• Normalization for starting position
– treat a chain code as a circular sequence and redefine the
starting point so that the resulting sequence of numbers
forms an integer of minimum magnitude.
Initial starting point
33001122

33001122 00112233
30011223 01122330
00112233 11223300 First row gives the
01122330 12233001
11223300
Sort
22330011
normalized chain code
rows
12233001 23300112 00112233
22330011 33001122
23300112 30011223 MATLAB function: sortrows
 Differential Chain Code (DCC)
 We can normalize also for rotation by using the first
difference of the chain code instead of the code itself.

 The difference is obtained by counting the number of

direction changes (in counterclockwise) that separate two
adjacent elements of the code i.e. taking the difference
between two adjacent pixels
Chain code : The first
Example: difference
1
01 1
02 2
2 0 03 3
23 1
3 20 2
21 3
 Example: Chain code : The first
difference
1 01 1
02 2
03 3
2 0 23 1
20 2
21 3
3

Assuming the first difference code represent a closed path,

rotation normalization can be achieved by circularly shifting
the number of the code so that the list of numbers forms the
smallest possible integer.
Example:
- a chain code: 10103322
- The first difference = 3133030
- Treating code as a circular sequence to normalize w.r.t. starting
point, then the first element of the difference is computed by
using the transition between the last and first components of the
chain, we get the first difference = 33133030
 Another Example !

The first difference of smallest magnitude — obtained by

treating the resulting array as a circular array and rotating it
cyclically until the resulting numerical pattern results in the
smallest possible number — is known as the shape number of the
contour.
The shape number is rotation invariant and insensitive to the
starting point used to compute the original sequence.
 Exercise
• For the following T-shape object shown in the middle of the
figure:
a. Write the 4-directional chain code (use the top-left corner as
the starting point);
b. Calculate its shape number (i.e., normalized differential chain
code);
c. Calculate the shape number of the T-shape object after 180o
rotation as shown at the right side of the figure.
Differential Chain Code: Example

Original Chain code Differential Chain code

 Chain Code: Summary
– Representation of binary images
– Traversing the edges in steps and encoding each step
– Contains 4 or 8 codes
– Each represents a direction between two pixels
– Coding depends on the direction
– The IPT does not have built-in functions for
computing the chain code, Freeman code, and
shape number of a contour.
 Boundary Descriptors
Length of a boundary
Diameter
Eccentricity
Circularity
 Length of a boundary
• the number of pixels along a boundary
give a rough approximation of its length
 Diameter
 Maximum distance between any two
points on the boundary
 The line formed by this two points is
called the major axis of the boundary

Diam( B)  max[ D( pi , p j )]
i, j

• D is a distance measure
• pi and pj are points on the boundary B
 Eccentricity
 A quantity defined for a
conic section which can be
given in terms of
semimajor axis a (i.e. one
half of the major axis) and
semiminor axes b.
 major axis = the line
connecting the two
extreme points that
comprise the diameter
 minor axis = the line
perpendicular to the
major axis
 Circularity
As indicated by its name, measures how a
circular an object is. For, example circularity of
a perfect circle would be 1.
 Feature Matching

Find big circles

 Regional Descriptors
• area
• perimeter
• Bounding box
• Compactness
• Convex Hull
• topological descriptors
 Regional Descriptors (contd.)
area = The number of pixels contained
within object’s boundary
perimeter = length of its boundary,
Bounding box: minimal axis-parallel
rectangle that encloses all the points of the
region
Compactness = (perimeter)2/area
– Insensitive to scale changes
– Insensitive to orientation
 Convex Hull
Convex hull: the smallest convex polygon within
which all points in the region fit.
 Convex Hull (contd.)
• Convexity: relationship between the
length of the convex hull and the
perimeter of the region

• Density: the ratio between the area of

the region and the area of the convex
hull
 Topological descriptors
 Topology is the study of properties of a figure that
are unaffected by any deformation, as long as there
is no tearing or joining of the figure.
 For example, following figure shows a region with
two holes.
 Topological descriptors (cont.)
 Thus if a topological descriptor is defined by the
number of holes in the region, this property
obviously will not be affected by a stretching or
rotation transformation.

 In summary, topological descriptors provide an

additional feature that is often useful in
characterizing regions in a scene
 Euler Number
 The number of holes and connected
components in a figure can be used to
define the Euler number as follows:
E=C-H
Where,
E = Euler number
C = number of connected region
H = number of holes
 Euler number: Example

EN=0 EN=-1

EN=-3

MATLAB function: bweuler returns the Euler number for

the binary image
 Topological Descriptor: Application
 Following figure shows an image of Washington DC area taken
by NASA satellite and its corresponding thresholded image.

 The objective of this example is to show how connected

components can be used to “finish” the segmentation.

River
Topological Descriptor: Application (cont.)
 If we would like to segment the river using this image,
than as illustrated by the thresholded image in the
previous slide, that it would be impossible to segment
the river by itself without other regions of the image also
appearing in the thresholded result.

 Analysis of thresholded image showed that image has

1591 connected components and its computed Euler
number is 1552.

 It can be deduced that number of holes are 39.

Topological Descriptor: Application (cont)
Following figure shows the largest connected
components, which is the desired result.
 Image Representation and
Description: Summary
Objective:
To represent and describe information
embedded in an image in other forms that are
more suitable than the image itself.
Benefits:
- Easier to understand
- Require fewer memory, faster to be processed
- More “ready to be used”
 Image Representation and
Description: Summary (contd.)
• What kind of information we can use?
- Boundary, shape
- Region
- Relation between regions
 Matlab Commands
• bwboundaries: traces the exterior boundaries of objects,
as well as boundaries of holes inside these objects, in the
binary image

• bwperim: Find perimeter of objects in the binary image

• bweuler: computes Euler number of binary image

• bwlabel: Label connected components in 2-D binary

image

• regionprops: measures a set of properties for each

labeled region
 Matlab Commands (contd.)
• bwarea: estimates the area of the objects in
binary image
• bwareaopen: Morphologically open binary
image i.e. remove small objects
 What Next?
We have looked at different kinds
of Image Representation &
Description techniques
Next time we will start to look at
Object recognition