ROBOT VISION, IMAGE

PROCESSING AND ANALYSIS

Presented By:

Sumit Mane(162110017)
Akhilesh Gupta(162110001)
Kunal Karnik(162110013)
 CONTENTS
What is a robot?
Why robot vision is required?
What is robot vision system?
Its types
Actual examples
Applications
Future scope
What is a robot?

"A reprogrammable, multifunctional manipulator designed to move

materials, parts, tools or specialized devices through various
programmed motions for the performance of a variety of tasks"

-Robot Institute of America

Robot Structure
 Why robot vision is required?
Comparing robot with human

Comparing robot anatomy with human anatomy

To identify and accurately determine the positions of all

relevant three dimensional objects within the robot work
place.

Input to the system Smart Robots

 What is robot vision system?
Defined as the process of extracting, analyzing, and
interpreting information from images of a 3-D world
Fundamental Tasks of Robot Vision

Image extraction/transformation

1. Edge detection
2. Corner detection
3. Blob detection
Image analysis
- Analyzing the extracted image
- Physical understanding of image

Image interpretation
- Concluding the image
- Performing in accordance

Example:- Picking a round shaped object

 IMAGE PROCESSING
VS
IMAGE ANALYSIS
-Image processing relates to preparation of
an image for later analysis and use.
-Image analysis is the collection of process in
which a captured image that is prepared by
image processing is analyzed in order to
extract information about image and
identifying the object.
 WHAT IS IMAGE??
AN image is a representation of real scene,
either in black and white or in color, and
either in print form or in a digital form.
Image can be 2D and 3D.
Depending upon color composition image can
be divided into
1.Binary image
2.Grey image
3.Color Image
TWO DIMENSIONAL AND THREE
DIMENSIONAL IMAGES
ACQUISITION OF IMAGES
ANALOG CAMERA
-Analog camera are the cameras that
transforms the image in analog electric signal.
DIGITAL CAMERA
-Digital camera are those camera that
transforms the image directly in digital form
i.e. binary number form.
ANALOG VS DIGITAL
ANALOG CAMERA

VIDICON CAMERA
DIGITAL CAMERA
FREQUENCY DOMAIN VS SPATIAL
DOMAIN
SAMPLING AND
QUANTIZATION
The intensity of light at each pixels are read
the more pixel that are present and
individually read, the better is the resolution
of the camera and image this technique is
called sampling.
The voltage charge read at each pixel is an
analog value and must be digitized.
Digitization of light intensity value at each
point is called quantization.
IMAGE PROCESSING TECHNIQUE
HISTOGRAM
THRESHOLDING
NOISE REDUCTION
EDGE DETECTION
SEGMENTATION
BINARY MORPHOLOGY OPERATION
GRAY MORPHOLOGY OPERATION
 HISTOGRAM
Histogram is presentation of total number of pixel of
image at each gray level.
 THRESHOLDING
Thresholding is the process of dividing an image into different
portions by picking a certain grayness level as a threshold.
 NOISE REDUCTION
Removal of noise caused due to various factors like faulty image,
environmental effect and etc.
 SEGMENTATION
Segmentation is a generic name for number of techniques that divide
the image into segments of its constituent.
Morphological Image Processing
Morphology a branch in biology that deals with
the form and structure of animals and plants.
Mathematical Morphology as a tool for
extracting image components, that are useful in the
representation and description of region shape.
The language of mathematical morphology is Set
theory.
Unified and powerful approach to numerous image
processing problems.
In binary images , the set elements are members of the

2-D integer space, where each element (x,y) is a

coordinate of a black (or white) pixel in the image.
Basic Concepts in Set Theory
Subset

Union

Intersection

disjoint / mutually exclusive

Complement
Difference
Reflection
Translation
Logic Operations Involving Binary
Pixels and Images
The principal logic operations used in image processing
are: AND, OR, NOT (COMPLEMENT).
These operations are functionally complete.
Logic operations are preformed on a pixel by pixel basis
between corresponding pixels (bitwise).
Other important logic operations :
XOR (exclusive OR), NAND (NOT-AND)
Logic operations are just a private case for a binary
set operations, such : AND Intersection , OR Union,
NOT-Complement.
Structuring Elements
The structuring element consists of a pattern
specified as the coordinates of a number of
discrete points relative to some origin.
Normally Cartesian coordinates are used and
so a convenient way of representing the
element is as a small image on a rectangular
grid.
The origin is marked by a ring around that
point. The origin does not have to be in the
center of the structuring element, but often it
is.
Structuring Elements

Examples of Structuring Elements

As suggested by the figure, structuring
elements that fit into a 33 grid with its
origin at the center are the most
commonly seen type.
Dilation
Dilation is used for expanding an element A by using
structuring element B
Dilation of A by B and is defined by the following
equation:

This equation is based 0n obtaining the reflection 0f B

about its origin and shifting this reflection by z.
The dilation of A by B is the set of all displacements z,
such that and A overlap by at least one element.
Based
On this interpretation the equation of can be rewritten
as:
Dilation Example 1
Dilation Example 2

The image shows the result of dilating this

image with a 33 square structuring element.
Note that although the noise has been
effectively removed, the image has been
degraded significantly.
9.2.2 Erosion
Erosion is used for shrinking of element A by
using element B
Erosion for Sets A and B in Z2, is defined by the
following equation:

This equation indicates that the erosion of A

by B is the set of all points z such that B,
translated by z, is combined in A.
Erosion Example 1
9.2.2 Erosion Example
2
9.2.2 Erosion Example
2
Erosion and Dilation
summary
Opening And Closing
Opening smoothes contours , eliminates
protrusions

Closing smoothes sections of contours, fuses

narrow breaks and long thin gulfs, eliminates
small holes and fills gaps in contours

These operations are can be applied few

times, but has effect only once
Opening And Closing
Opening
First erode A by B, and then dilate the result
by B
In other words, opening is the unification of all
B objects Entirely Contained in A
Opening And Closing
Closing
First dilate A by B, and then erode the result
by B
In other words, closing is the group of points,
which the intersection of object B around them
with object A is not empty
Use of opening and closing for
morphological filtering
Boundary Extraction
First, erode A by B, then make set difference
between A and the erosion
The thickness of the contour depends on the
size of constructing object B
Boundary Extraction
Region Filling
This algorithm is based on a set of dilations,
complementation and intersections
p is the point inside the boundary, with the
value of 1
X(k) = (X(k-1) xor B) conjunction with
complemented A
The process stops when X(k) = X(k-1)
The result that given by union of A and X (k), is
a set contains the filled set and the boundary
Gray-Scale Images
In gray scale images on the contrary to binary
images we deal with digital image functions of
the form f(x,y) as an input image and b(x,y) as
a structuring element.
(x,y) are integers from Z*Z that represent a
coordinates in the image.
f(x,y) and b(x,y) are functions that assign gray
level value to each distinct pair of coordinates.
For example the domain of gray values can be
0-255, whereas 0 is black, 255- is white.
9.6 Gray-Scale Images
Dilation Gray-Scale
In this case, each pixel is replaced by the
value of lightest pixel in its 3x3 neighborhood,
known as a max operator and effectively
dilates the object.
The result is dependent on which cell in the
mask are 0 or 1
Grey morphology dilation removes dark
bridges between light objects.
Erosion Gray-Scale
In this case, each pixel is replaced by the
value of the darkest pixel in its 3x3
neighborhood, known as min operator and
effectively erodes the object.
Of course, the result is dependent on which
cells in mask are 0 or 1.
Grey morphology erosion removes light
bridges between objects.
 Dilation & Erosion Gray-Scale

Over Applying the Filter Filter Demonstration

9.6.3 Opening And Closing
Similar to the binary algorithms
Opening
Closing
In the opening of a gray-scale image, we
remove small light details, while relatively
undisturbed overall gray levels and larger
bright features
In the closing of a gray-scale image, we
remove small dark details, while relatively
undisturbed overall gray levels and larger
dark features
9.6.3 Opening And Closing
Opening a G-S picture is describable as
pushing object B under the scan-line graph,
while traversing the graph according the
curvature of B
9.6.3 Opening And Closing
Closing a G-S picture is describable as
pushing object B on top of the scan-line
graph, while traversing the graph according
the curvature of B
The peaks are usually
remains in their
original form
9.6.3 Opening And Closing
 Image Analysis
It is a collection of operations and techniques
that are used to extract information from
images.
Among these operations and techniques are:
Object recognition
Feature extraction

Analysis of position, size, orientation

Extraction of depth information

Generally it is assumed that image processing

routines have already been applied to the
image to prepare the image.
Object Recognition by
Features
Objects in the image can be identified by
morphological features such as area,
perimeter, number of holes, eccentricity etc.
In many cases the information extracted is
compared with saved information about
object, which may be in the look up table.
We next discuss a few techniques and
different features that may be used for object
recognition
Features used for Object
Identification
Colour and Texture
Morphological Characteristics
Perimeter, area, diameter
Number of holes or flanges etc.
Moment equations
Objects aspect ratio i.e. ratio of width to
length
Colour and Texture
The average, maximum or minimum grey
levels may be used to identify different parts
or objects in image.
Different colours and textures give different
grey levels. If the average, maximum, or
minimum grey levels of the object are found,
the object can be recognized.
In some cases, even the presence of one
particular grey levels may be enough to
recognize a part.
Morphological
characteristics
The perimeter, area, diameter of an object
may be used to identify the object.
The perimeter can be found by first applying
edge detection routine and then counting no
of pixels on perimeter
Number of holes, projections etc. can also be
used to identify the object.
Objects aspect ratio
Aspect ratio is the ratio of
width to the length of a
rectangle enclosed object.
Aspect ratio can be used
to detect the object.
However, aspect ratio
depends on the
orientation of the object.
Thus minimum aspect
ratio is generally used to
identify the object.
Template Matching
Another technique of object recognition is
model or template matching.
If a suitable line drawing of scene is found,
the total no of lines(sides), vertices, and
interconnections can be matched to a model.
The prior knowledge of model is needed for
matching
Thus, if the object is different than from the
models, they will not match and the object will
not be recognized
Computed Tomography
(CT)
Tomography is a technique of determining the
distribution of material density in the part
being examined.
In CT, a 3D image of object is reconstructed
from large number of 2D images of the
material density taken by scanning techniques
such as X-Rays or Ultrasonics.
The data is stored in a computer and
subsequently reduced to Three dimensional
image of parts density distribution.
Range detection and Depth
Analysis
Humans can extract useful information from
details such as changing size of similar
elements, vanishing lines, shadows, and the
changing intensity of textures and shades.
Techniques used for range and depth
detection are
Stereo imaging
Scene analysis by shading and sizes
Specialized lighting
Stereo Imaging
If two images of the same scene are formed
by two different cameras, then the relative
depths of different points from the image
plane can be extracted by comparing two
images.
Humans do the same automatically by
combining the two images forming 3D image
information.
The stereo image is considered as 2.5 D
image: many more images are required to
form a 3 D Image.
The Accuracy of
depth measurement
in stereo imaging
depends on the
angle between two
images.
To improve accuracy
and reduce
computation tome,
multiple images of
same scene can be
Scene Analysis with
Shading
Shading is the relationship between the
orientation of the object and the reflected
light.
Smoothly changing intensity of shades can be
indirectly used for extracting information
about the depth and shape of object.
Depth measurement requires prior knowledge
of reflectance properties of the object and
exact knowledge of the light source. Hence
the utility is limited and difficult.
Applications
In conjunction with Robotic operations and
robots, this can be used in many different
applications.
Some of the applications are
Vision guided pick and place robots
Palletizing robots
Inspection
Navigation
Identification of parts
Assembly operations.
Thank
You!!