Robotika

Pertemuan 6 : Introduction to Machine Vision

Prodi Sistem Komputer

Universitas Komputer Indonesia
 What is Computer Vision?
Image Processing
Computer Graphics

• Three Related fields

• Image Processing: Changes 2D images into other 2D images
• Computer Graphics: Takes 3D models, renders 2D images
• Computer vision: Extracts scene information from 2D
images and video
• e.g. Geometry, “Where” something is in 3D,
• Objects “What” something is”
• What information is in a 2D image?
• What information do we need for 3D analysis?
90
Computer Vision

horizon

Computer Vision
Pertemuan 6 : Introduction to machine Vision 2
 Machine Vision
• 3D Camera vision in general environments hard
• Machine vision:
• Use engineered environment
• Use 2D when possible
• Special markers/LED
• Can buy working system!
• Photogrammetry:
• Outdoors
• 3D surveying using cameras

Pertemuan 6 : Introduction to machine Vision 3

 Vision
• Full: Human vision
• We don’t know how it works in detail
• Limited vision: Machines, Robots, “AI”
• What is the most basic useful information we can
get from a camera?
• Location of a dot (LED/Marker) [u,v] = f(I)
• Segmentation of object pixels v
• All of these are 2D image plane measurements!
• What is the best camera location? u
• Usually overhead pointing straight down
• Adjust cam position so pixel [u,v] = s[X,Y].
• Pixel coordinates are scaled world coord
Y

Pertemuan 6 : Introduction to machine Vision 4

 Tracking LED special markers
• Put camera overhead pointing straight down on v
worktable.
• Adjust cam position so pixel [u,v] = s[X,Y]. Pixel
coordinates are scaled world coord u
• Lower brightness so LED brighterest
• Put LED on robot end-effector
• Detection algorithm:
• Threshold brightest pixels I(u,v)>200
• Find centroid [u,v] of max pixels Camera
• Variations:
• Blinking LED can enhance detection in ambient light.
• Different color LED’s can be detected separately from Y
R,G,B color video. LED

Pertemuan 6 : Introduction to machine Vision 5

Commercial tracking systems

Polaris Vicra infra-red system MicronTracker visible light system

(Northern Digitial Inc.) (Claron Technology Inc.)

Pertemuan 6 : Introduction to machine Vision 6

Commercial tracking system
Images acquired by the Polaris Vicra infra-red stereo system:

left image right image

Pertemuan 6 : Introduction to machine Vision 7

 IMAGE SEGMENTATION
•How many “objects” are there in the image below?
•Assuming the answer is “4”, what exactly defines an
object?
Zoom In

Pertemuan 6 : Introduction to machine Vision 8

 8 BIT GRAYSCALE IMAGE

200

150

100

50

Pertemuan 6 : Introduction to machine Vision 9

 GRAY LEVEL THRESHOLDING

Objects

Set threshold here

11/12/2021 Introduction to Machine Vision

Pertemuan 6 : Introduction to machine Vision 10
 BINARY IMAGE

Pertemuan 6 : Introduction to machine Vision 11

 What are some examples of form parameters
that would be useful in identifying the objects
in the image below?

Binari Image :
Black = 0
White = 1

Pertemuan 6 : Introduction to machine Vision 12

 3D MACHINE VISION SYSTEM

XY Table

Laser Projector
Digital Camera

Field of View

Plane of Laser Light Granite Surface Plate

P(x,y,z)

Pertemuan 6 : Introduction to machine Vision 13

 3D MACHINE VISION SYSTEM

11/12/2021 Introduction to Machine Vision

Pertemuan 6 : Introduction to machine Vision 14
 3D MACHINE VISION SYSTEM

Pertemuan 6 : Introduction to machine Vision 15

 Morphological processing (Erosion)

Pertemuan 6 : Introduction to machine Vision 16 16

 Morphological processing (Erosion)

Pertemuan 6 : Introduction to machine Vision 17 17

 Morphological processing (Dilation)

Pertemuan 6 : Introduction to machine Vision 18 18

 Image Processing

Filtering: Noise suppresion Edge detection

Pertemuan 6 : Introduction to machine Vision 19

 Image filtering

0 0 0 0 0 0
Mean filter 0 0 0 0 0 0
0 0 0 0 0
0 0 0 0 0 0 0 22 22 22 0 0

=
0 0.11 0.11 0.11 0
0 0 200 0 0 0 0 22 22 22 0 0

0
0

0
0

0
0

100

0
0

0
0

0
* 0

0
0.11 0.11 0.11

0.11 0.11 0.11

0 0 0
0

0
0

0
22

0
33

11

11
33

11

11
11

11

11
0

Input image f Filter h Output image g

Pertemuan 6 : Introduction to machine Vision 20

 Convolution
To accomplish convolution of the whole image,
p1,1 p1,2 p1,3 p1,4 p1,5 p1,6 we just Slide the mask
p2,1 p2,2 p2,3 p2,4 p2,5 p2,6
p3,1 p3,2 p3,3 p3,4 p3,5 p3,6
p4,1 p4,2 p4,3 p4,4 p4,5 p4,6 m1,1m1,2m1,3

p5,1 p5,2 p5,3 p5,4 p5,5 p5,6 m2,1m2,2m2,3

m3,1m3,2m3,3
p6,1 p6,2 p6,3 p6,4 p6,5 p6,6
Original Image Mask

C4,2 C4,3 C4,4

Image after convolution

Pertemuan 6 : Introduction to machine Vision 21

 Gaussian filter

Compute empirically

* =
Filter h
Input image f Output image g

Pertemuan 6 : Introduction to machine Vision 22

 Convolution for Noise Elimination

--Noise Elimination
The noise is eliminated but the operation causes loss of
sharp edge definition.
In other words, the image becomes blurred
Pertemuan 6 : Introduction to machine Vision 23
 Convolution with a non-linear mask
Median filtering
There are many masks used in Noise Elimination

Median Mask is a typical one

The principle of Median Mask is to mask some sub-image, use the median of the
values of the sub-image as its value in new image

J=1 2 3

I=1 23 65 64 Rank: 23, 47, 64, 65, 72, 90, 120, 187, 209

2 120 187 90
47 209 72 median
3

Masked Original Image

Pertemuan 6 : Introduction to machine Vision 24

Median Filtering
 Edge detection using the Sobel operator

In practice, it is common to use:

-1 0 1 -1 -2 -1
-2 0 2 0 0 0
-1 0 1 1 2 1

Magnitude:

Orientation:

Pertemuan 6 : Introduction to machine Vision 26

 Sobel operator

Original Magnitude Orientation

Pertemuan 6 : Introduction to machine Vision 27

 Effects of noise on edge detection /
derivatives
• Consider a single row or column of the image
• Plotting intensity as a function of position gives a signal

Where is the edge?

Pertemuan 6 : Introduction to machine Vision 28
 Solution: smooth first

Where is the edge? Look for peaks in

Pertemuan 6 : Introduction to machine Vision 29
 Vision-Based Control (Visual Servoing)

Initial Image

User

Desired Image

Pertemuan 6 : Introduction to machine Vision 30

 Vision-Based Control (Visual Servoing)

: Current Image Features

: Desired Image Features

Pertemuan 6 : Introduction to machine Vision 31

 Vision-Based Control (Visual Servoing)

: Current Image Features

: Desired Image Features

Pertemuan 6 : Introduction to machine Vision 32

 Vision-Based Control (Visual Servoing)

: Current Image Features

: Desired Image Features

Pertemuan 6 : Introduction to machine Vision 33

 Vision-Based Control (Visual Servoing)

: Current Image Features

: Desired Image Features

Pertemuan 6 : Introduction to machine Vision 34

 Vision-Based Control (Visual Servoing)

: Current Image Features

: Desired Image Features

Pertemuan 6 : Introduction to machine Vision 35

 u,v Image-Space Error

u
: Current Image Features s t
: Desired Image Features sd
E= [ à ]
One point
E = [y 0 à y ã] e t = s t − sd
Pixel coord
ô õ ô õã
yu yu
E= à
yv yv e t d = s t d − sd = 0
Many points
2 3ã 2 3
y.1 y.1
E = 4 .. 5 à 4 .. 5
Pertemuan 6 : Introduction to machine Vision y16 y16 0 36
 Other (easier) solution: Image-based motion control

Note: What follows will work for one or two (or 3..n) cameras.
Either fixed (eye-in-hand) or on the robot.
Here we will use two cam

Achieving 3d tasks
Motor-Visual function: y=f(x)
via
2d image control
Pertemuan 6 : Introduction to machine Vision 37
 Vision-Based Control

Image 1 Image 2

Pertemuan 6 : Introduction to machine Vision 38

 Vision-Based Control

Image 1 Image 2

Pertemuan 6 : Introduction to machine Vision 39

 Vision-Based Control

Image 1 Image 2

Pertemuan 6 : Introduction to machine Vision 40

 Vision-Based Control

Image 1 Image 2

Pertemuan 6 : Introduction to machine Vision 41

 Vision-Based Control

Image 1 Image 2

Pertemuan 6 : Introduction to machine Vision 42