VISUAL SERVOING

ROBOT FOR PICK AND

PLACE UNDER PARTIAL
OCCLUSION – II
Supervised By: Presented By:
Dr. Abbas Zilqurnain Naqvi Danish 2017-MC-01
Usama Ali 2017-MC-08
M Shehryar 2017-MC-27
 • Abstract
• Introduction to the Problem Statement
• Objectives and Scope
CONTENTS • Literature Review
• Design Methodology
• Progress
• Execution Plan
 ABSTRACT

• The project is aimed at the Control of

Robotic Manipulator for Pick and Place
objects, may be under Partial Occlusion,
using visual feedback.
 Visual Servoing
Visual Servoing is a technique which uses
feedback information extracted from
a vision sensor (visual feedback) to control
the robot’s end-effector, relative to the
target.
I N T RO DU C TI ON
TO P ROB L EM STAT EM EN T
 Occlusion
It occurs either due to one object is hidden
by another object or an object is hidden by
some component of the background.
During occlusion, an ambiguity occurs in
the objects and their features.
I N T RO DU C T I O N TO
P RO B L EM STAT EM E N T
 OBJECTIVES

OBJECT OCCLUSION ROBOT CONTROL TARGET PICK

DETECTION HANDLING AND PLACE
 SCOPE OF
PROJECT

• Handling 30-40%
occlusion of objects.
• Deployed on generalize
objects for pick and
place.
• Target Picking of
objects.
METHODOLOGY
 METHODOLOGY

Image Segmentation:
The process of partitioning a
digital image into multiple
segments (sets of pixels also
known as image object). The
goal of segmentation is to
simplify and/or change the
representation of an image into
something that is more
meaningful and easier to
analyze.
 METHODOLOGY

Image Segmentation
• Semantic Segmentation:
It predicts the pixel wise class
labels for objects in an image.
• Instance Segmentation:
It predicts the pixel wise class
label and mask for each
instance in an image.
 LITERATURE REVIEW

K Wada et al. [3] have presented instance

segmentation, in which occluded region of each
instance is segmented as the visible one, seeing it as
multi-class. Mask R-CNN is used for bounding box
based object detection and then predicting instance
mask.
 LITERATURE REVIEW

Kei Okada et al. [4] devised the collaboration of instance-level reasoning of Instance
segmentation and image-level reasoning of Semantic Segmentation. Instance
Occlusion Segmentation for multi-class masks (Box, Class, Mask) and Semantic
Occlusion Segmentation for pixel-wise Labeling.
FLOW CHART
 PROGRESS TILL NOW…

• Kinect Integration with Python.

• Serial Communication of Inverse Kinematics data from Python to Arduino.

PROGRESS
VIDEO
GANTT CHART
 REFERENCES

1. https://petercorke.com/rvc/v-visual-servoing/15-visual-servoing/
2. https://datascience.stackexchange.com/questions/52015/what-is-the-difference
-between-semantic-segmentation-object-detection-and-insta
3. https://ieeexplore.ieee.org/document/8593690
4. https://ieeexplore.ieee.org/document/8793783