Subject Introduction

Robotics and Automation

ECE2008

Dr. Om Prakash Sahu

Sr. Assistant Professor G-II (SENSE)
Centre for Innovation and Product Development
Vellore Institute of Technology, Chennai
 What is Sensor
Contents
 Application of Sensors

 Type of Industrial Robots

 Use of Sensors in Robot

 Sensor Integration

Classification of Sensors
Sensors DAQ system for Control
Example for Sensor Integration in Robotic hand
Experimental Demonstrations
 Applications and Future scope of Robotics Sensors
Opportunities in Robotics
Sources and Links
Application of Sensors
Type of Industrial Robots
 Use of Sensor in Robot

1. Industrial applications 2. Pick and place operation 3. General material handling

4. Role in hazardous environments 5. HRI 6. Prosthetic Hand Arm

NAO: A typically complex humanoid robot system
NAO: A typically complex humanoid robot system
 Examples of Artificial Sensors

Clockwise from the top left: Microsoft Kinect® and Intel Realsense® (vision and distance), Sony D-Link DCS-5222L® and
AVT GT® (vision), Syntouch BioTac® and ATI Nano 43® (touch), sound sensor LM393® and 3Dio Free Space Pro II®
Binaural Microphone (audition), proximity sensor Sharp GP2Y0A02YK0F®, Laser SICK®, Hokuyo URG®, and proximity
sensor SICK CM18-08BPP-KC1® (distance). Note that Intel Realsense® and Microsoft Kinect® provide both the senses of
vision and of distance.
 Sensor Integration in Industrial Hand

Gripper
or
Robotic Hand
or
End-effector
or
SCARA ROBOT EOAT
 Gripper or
Hand
Sensor Integration
Sensor-1

Sensor-2

Sensors integrated Gripper

Sensor-3

Sensor-4
Sensor-… n
Sensor-7
Sensor-5
Sensor-6
 Sensor Integration
• Observe the unstructured parts and workspace.

• Identified the part and its properties.

• And perform the task or operation like pick and place.

Workspace
 Sensor Integration
• Observe the unstructured environment and workspace.

• Identified the part and its properties.

• And perform the task or operation like pick and place.

Workspace
 Sensor Integration
• Observe the unstructured environment and workspace.

• Identified the part and its properties.

• And perform the task or operation like pick and place.

Workspace
 Classification of Robotic Sensors

 Proprioceptive Sensors
 Exteroceptive Sensors

 Exproprioceptive Sensors
 Classification of Proprioceptive Sensors

Motion
Velocity Sensor Optical
Sensors Absolute
Encoder
Encoder
Position Sensors
Proprioceptive Sensors
Magnetic
Sensors Sensor
LVDT
Acceleration Sensor
Sensor
Mechanical
RVDT Sensor
Joint Angle Sensor
Sensor

To Measure : Velocity, Position and Acceleration.

 Classification of Exteroceptive Sensors
Proximity Sensors

Gyroscope Sensor Inductive Sensor

Proximal
Sensors
Magnetic Sensor Capacitive Sensor

Medial Optical Sensor

Exteroceptive Sensors Optical Sensor
Sensors
Infrared Sensor
Distal
3D Vision Sensor
Sensors
Electromagnetic
2D Vision Sensor
Vision Sensors
Image Sensor
Ultrasonic Sensors

Laser Sensors Lidar Sensor

To Measure : The position of the object presents

 Classification of Exproprioceptive Sensors

Force/ Torque
Tactile Sensor
Sensors

Vulcan Tactile
Contact Sensor
Exproprioceptive Sensors
Limit Switch
Sensors Sensor
Switch
Surface Sensor
Array
Sensor LTS
Piezo-Film Sensor
Contact Sensor

To measure: The position of the Robot body or parts

 Classification of Sensors
Other sensors
Proximity 11%
Sensor Vision
3% Sensor
24%

Infrared laser
8%

Touch
sensors Tactile
7% Sensor
16%
Optical
Sensor
10%
Force Laser
Sensor sensors
10% 11%
Various type of sensors used in robotic applications.
Department of Industrial Design, NIT Rourkela
 Classification of Sensors

Department of Industrial Design, NIT Rourkela

 Vision sensor or camera

Motion detectors : are based on infrared light, ultrasound or microwave/radar technology,

Image sensors are digital cameras,
 Ultrasonic sensor

Detect distance
 Capacitive Proximity Sensors

Detecte : presence of the Metallic object

 Inductive Proximity Sensors

Tracking the objects

 Force/Torque sensor

The Force/Torque sensors are used to measure the force or torque during the operation of
assembly operation such as material handling, screwing and pressing in page-in-holes.
 LTS sensor

Tactile light touch switch (LTS) sensor gives the information

about whether the robot fingers are in contact with the object
or not.
Sensors DAQ system
 Control Process using (DAQ) Data Acquisition
System

A DAQ device acts as the interface between a computer and

signals from the outside world by digitizing incoming analog
signals to be computer readable.
 Control Process using (DAQ) Data Acquisition
System
Sensor -1

Sensor -2
Signal processing
and Graphical
Sensor-3 Signal interfacing with
conditioning system
Sensor- 4

Sensor- 5 Power Software

Supply Programming
Sensor -6 with Control
platform

Robot protocols

Material Handling Robot Limbs

using
ROBOT Limbs
 Sensors (DAQ) Data Acquisition Control System

1. NI-DAQmx driver software

2. Software for data acquisition
3. Programming environments for data acquisition

1. NI-DAQmx driver software 3. Programming environments

• NI-DAQmx: Reduce Development • FlexLogger: FlexLogger
• DAQ Assistant 2. Software for data acquisition is application software
• NI-DAQmx Create Virtual Channel • LabVIEW for quick sensor
• NI-DAQmx Trigger • LabWindows/CVI configuration and data
• NI-DAQmx Timing • Measurement Studio logging of mixed signals
• NI-DAQmx Start Task to verify
• NI-DAQmx Read electromechanical
• NI-DAQmx Write systems, all without
• NI-DAQmx Wait Until Done programming
• NI-DAQmx Clear Task • OPenCV Library
• NI-DAQmx Properties • Python
• MATLAB
• ROS
Control Integration with ROS System
 Considered Assembling
Experimental Parts
Demonstrations

A B C

D E F

G H I Integrated hand along with Industrial robot

 Developed Algorithm
Experimental Demonstrations
Start

Vision sensor image file

Camera-1

Decide Control Points and Target Points Camera-2 Camera-3

(Template matching)
Picking
Space
Determining the control point in the next frame Mitsubishi
using optical flow Robot

Get visual deviation

Camera-5
Placing
Determine input command (Visual deviation Space
Camera-6
multiplied by gain)

Enter the command to operate the

robot workspace
Camera-4
Implementation of the developed system for
various assembly operations
Camera setup and arrangements
End
 Picking Methods Experimental Demonstrations

Marking Points
at Part-A for
Picking

Series of
flows
Algorithm
Camera-1 on Z- axis
Eye in hand

Marking Points at Part-A for Picking Camera-1 (eye in hand)

on Z- axis
 Experimental Demonstrations
8
7 9
0 3
8
1 4 7 9

5
2 0 1 2 0 2

Camera-1 on X- axis Camera-2 on Y- axis Camera-3 on Z- axis

Force(N) Moment (Nm)

x y z 0.3

10 0.2
0.1
0
0 10 20 30 40 50 60 70 0
-10 0 10 20 30 40 50 60 70
-0.1
-20 -0.2

-30 -0.3

-40 a b c

Picking assembling parts-A

 Experimental Demonstrations

P3 P4
P4 P3
P3
P4
P1 P2 P1 P2 P1
P2

Camera-2, Camera-3 Camera-4, Camera-5 Camera-1 (eye in hand)

on X- axis on Y- axis on Z- axis

Placing Methods P1, P2, P3 & P4, Decided Points for Placing
 Experimental Demonstrations

P1, P2, P3 &

P4, Decided
Points for
Placing

Series of
flows
Algorithm

Camera-2, Camera-3 Camera-4, Camera-5 Camera-1 (eye in hand)

on X- axis on Y- axis on Z- axis
Template matching for Placing
 Placing assembling parts-A

8
7
7 9 0 9
1
0 2 2

Camera-4 on X- axis Camera-5 on Y- axis Camera-6 on Z- axis

Force (N) Moment (Nm)

10 0.4
0.3
0
0 10 20 30 40 50 60 70 0.2
-10 0.1

-20 0
0 10 20 30 40 50 60 70
-0.1
-30
-0.2
-40 -0.3

x y z a b c
 Movie on assembling parts-A
Experiment in KAWA Lab
Integration With Lab-VIEW
Developed Robotic hand
• Industrial robots Applications and Future scope of Robotics Sensors
• Domestic or household robots
• Medical robots
• Telepresence
• Surgical Assistants
• Rehabilitation
• Military robots
• MAARS
• Gladiator
• Anbot
• Black Hornet
• Robobee
• Robosapien
• QRIO
• Aibo
• Under water Robots
• Bio Inspired Robots
• Humanoid Robotics
 Opportunities in Robotics

1. About Robotics Jobs in : ML, Vision Commuter, Machine Vision, Sensor

Integration.
2. Postdoc Fellowship.
3. Ph.D.
4. Researcher in ABB, Mitsubishi Electric, Mujin, Darpa, Kawasaki. Erriction.
5. International FDP Certificates.
6. International Conference.
7. Special IssuesJournals.
8. International Workshop.
9. International FDP Certificates.
10.International Meeting on latest trends.
11.Journal Editors.
 Sources and Links

1. http://robotics-worldwide.1046236.n5.nabble.com/ : About Robotics Jobs,

Special IssuesJournals, PostDoc Fellowship, Phd, Meeting, Conference,
Workshop.

2. https://www.ni.com/en-in/shop/data-acquisition.html : About ow-cost data

acquisition devices for USB, PCI, and PCI Express.

3. https://reviewindependent.com/ : About Latest trends & Market in Sensors.

4. http://github.com/ : About Any kind of coding and programing.

5. http://mathworks.in/ : About mathematical Simulation and graphical

representation.
6. https://www.azosensors.com/
At Mitsubishi Electric Lab, Japan
Dr. Om Prakash Sahu
Motion Intelligent Laboratory,
Robotic Research Centre
Ritsumeikan University, Shiga, JAPAN