Robotics and Machine Vision

(MBE 4032)

Dr Pakpong Chirarattananon
Professor Dong Sun
Department of Biomedical Engineering
City University of Hong Kong
Pakpong.c@cityu.edu.hk; Office Y6606
medsun@cityu.edu.hk; Office: Y6719
1
Topics to be discussed

 Robotics
 Kinematics
 Dynamics
 Robot Control
 Mobile Robot
 Biomimetic Robotics
 Machine Vision
 Image Acquisition
 Image processing
2
Assessment Pattern

 Time: Week 13 (in class)

 Coursework 40%
20% Exercises (2 assessments)

20% lab work and reports (3 labs)

 Examination 60%

3
 Part I: Robotics

Prerequisite:
• ODE
• Control System
• Matlab/Mathematica
木牛流馬，三國時期諸葛亮發明
(1700 years ago, invented by Zhu Ge Liang)

5
BigDog (2008), Boston Dynamics
 Hollywood’s imagination robots…
I, Robot

Star Wars

Ironman
Avatar
 What is a robot?
• Definition of Robot
– Webster’s Dictionary
• An automatic device that performs functions ordinarily
ascribed to human beings
– Robotics Institute of American
• A robot (industrial robot) is a reprogrammable, multifunctional
manipulator designed to move materials, parts, tools, or
specialized devices, through variable programmed motions
for the performance of a variety of tasks.
– Wikipedia
• A robot is a mechanical or virtual artificial agent, usually an
electro-mechanical machine that is guided by a computer
program or electronic circuitry.
 What is a robot?
• By general agreement, a robot is
– A programmable machine that imitates the actions or
appearance of an intelligent creature – usually a
human.
• To qualify as a robot, a machine must be able to:
– Sensing and perception: get information from its
surroundings
– Carry out different tasks: Locomotion or manipulation,
do something physical – such as moving or
manipulating objects
– Re-programmable: can do different things
– Function autonomously and/or interact with human
beings
 The Three Laws of Robotics
 A robot may not injure a human being or,
through inaction, allow a human being to come
to harm.
 A robot must obey the orders given to it by
human beings, except where such orders would
conflict with the First Law.
 A robot must protect its own existence as long
as such protection does not conflict with the First
or Second Laws.
 Development Trend
1st stage
Imaginary world

1920s
Human’s dream
of replicating
themselves
 Development Trend
1st stage 2nd stage
Imaginary world Structured
environments
1920s
Human’s dream 1950s
of replicating
Industrial
themselves
robots PUMA robot arm (1980s)
 Development Trend
1st stage 2nd stage 3rd stage
Imaginary world Structured Unstructured
environments environments
1920s
Human’s dream 1950s
of replicating
Industrial 1980s
themselves
robots
Field Robots

CMU Project
Field robots
 Development Trend
1st stage 2nd stage 3rd stage 4th stage
Imaginary world Structured Unstructured Human-centred
environments environments environments
1920s
Human’s dream 1950s
of replicating
Industrial 1980s
themselves
Robots
Field Robots 2000s
Service Robots

da Vinci Surgical System

Service robots

Entertainment robots
Robot applications
• Types of Robots
– Robot Manipulators
– Mobile Manipulators
– Locomotion
• Application in 4D environments
– Dangerous
– Dirty
– Dull
– Difficult
• 4A tasks
– Automation
– Augmentation
– Assistance
– Autonomous
Robot Applications
• Manufacture Industry
– Assembling
– Automation
• Biotechnology
– Micro/Nano
manipulation
– Sample handling
– Automated analysis
Robot Applications
 How are robots used?
• Industrial robots
– 70% welding and painting
– 20% pick and place
– 10% others
• Research focus on
– Manipulator control
– End-effector design
• Compliance device
• Dexterity robot hand
– Visual and force feedback
– Flexible automation
• Robot Manipulators
– Assembly, automation
• Field robots
– Military applications
– Space exploration
• Service robots
– Cleaning robots
– Medical robots
• Entertainment robots
 Architecture of Robotic Systems
Environmental Motion Mechanical
sensors planner Controller Structure

Configuration
sensor
• Mechanical Structure
– Kinematics model
– Dynamics model
• Actuators: Electrical, Hydraulic, Pneumatic, Artificial Muscle
• Computation and controllers
• Sensors
• Communications
• User interface
• Power conversion unit
Programming Robots
• On-Line programming
– Lead-Through
• Paint spraying, applying adhesive
to an irregular surface
• Taken through its operating cycle
manually
– Drive-Through
• Spot welding, machine loading
and unloading
• Controlled by inputs from a teach
pendant or keypad
• Off-Line programming
– Control the overall sequences of tasks
– Use high lever computer language
– Needs a large mount of computing
power, incorporates the use of
sensors
– Drive through programming used at
discrete points
Introductory Examples

• Robot manipulator
Introductory Examples
• Mobile robot
 Part II: Machine Vision
Visual Process:
The process of extracting, characterizing and interpreting
information from image of a 3D world
2D image ----> model of real world
Vision = geometry + measurements + interpretation
As one of the 5 senses of our human beings, vision is naturally an
important function to emulate/develop for robots. Although the current
study in machine vision is still in its infantry stage, successful
applications can be found in many areas including:
 Medical applications
 Space applications and weather forecasting
 Industrial applications
Industrial manufacturing cell with vision systems

29
System functions in manufacturing cell with vision system

30
Fundamental Steps

31
Fundamental Steps (con’t)

 Image acquisition – the process that yields a visual image

 Preprocessing – deals with techniques such as noise
reduction and enhancement of details
 Segmentation – the process that partitions an image into
objects of interest
 Description – deals with the computation of features (e.g.
size, shape) suitable for differentiating one type of object
from another
 Recognition & Interpretation
 the process that identifies these objects (e.g. wrench,
bolt, engine block)
 assigns meaning to an ensemble of recognized objects

32
Typical application areas of digital image processing are: (a) computer graphic,
(b) Image transmission, (c) image manipulation, (d) image analysis, e) scene analysis
33
Modules of Machine Vision System
A typical vision (image processing) system consists of the
following major functional modules

 illumination
 Image acquisition
 Image processing
 Image display

Typical scenario for an industrial image

processing system 34
Supplementary Readings
• Robotics
– Bajd, T., Mihelj, M.,
Lenarčič, J.,
Stanovnik, A., Munih,
M. 2010
• Robotics, Vision and
Control: Fundamental
Algorithms in
MATLAB
– Corke, Peter 2011