Part 07

Visual Information Interpretation: Image feature
Ji Hui
National University of Singapore
September 7, 2021
Ji Hui (National University of Singapore) Visual Information Interpretation: Image feature September 7, 2021 1 / 34
Image feature
A feature is defined as an ”interesting” part of an image

Distinct points
Salient edges
and many others
Starting point of many computer vision tasks
image alignment
3D scene reconstruction from multiple images
Object recognition
Tracking and navigation
and many others
Image edges
What are edges: Salient features of the scene
Edges are caused by a variety of factors:
Whatcauses
What causesanedges
edge?
[Source: K. Grauman]
Image gradient
Image gradient
rf = [ @f @f
@x , @y ]
Direction : ✓ = tan 1 ( @f @f
@y / @x )
q
2
Strength : krf k2 = | @f @f 2
@x + | @y | .
The gradient direction points to most rapid change in intensity
Differentiate a discrete image by finite difference operators

Equivalent to convolve image by a high-pass filter.
Edge vs gradient
Edge operators
Smoothing
Noise make direct calculation of gradient unreliable
Sobel operator
Zero-crossing
Consider a Gaussian filter h.
Edge point is located at the zero point
2D Gaussian edge detector
Laplacian of Gaussian
@2 @2
where r2 = @x2 + @y 2 .
Detection/Localization trade-off
More smoothing improves detection, but hurts localization
Less smoothing improves localization, but sensitive to noise
Canny edge detector
Basic procedure
Smooth image I with 2D Gaussian: G ⌦ I
Find local edge normal direction for each pixel
r(G ⌦ I)
~n =
kr(G ⌦ I)k
Compute edge magnitudes kr(G ⌦ I)k

Locate edges by finding the zero-crossings along the edge normal
@ 2 (G ⌦ I)
.
@~n2
Edge thresholding: only select ”strong edges”
⇢
1 if kr(G ⌦ I)(r)k > T
E(r) =
0 otherwise.
Edge thinning by mathematical morphology operator.

Canny edge detector
original image norm of gradients
thresholding thinning
Effect of scale in Canny edge detector
Effect of (Gaussian kernel spread/size)
original Canny with Canny with

The choice of depends on desired behavior
The choice of depends on desired behavior
large detects coarse scale edges
• large detects large scale edges
Small• small
detects detects
fine features
fine features
Finding lines in an image
Edge is not line

Question related to finding a line
Given points that belong to a line, what is the line?

How many lines are there?
Which points belong to which lines?
Hough transform: a voting scheme to find lines

Record vote for each possible line on which edge point lies.
Look for lines that get many votes
Voting: a general technique where we let the features vote for all models
that are compatible with it.
Cycle through features, cast votes for model parameters
Look for model parameters that receive a lot of votes
Finding lines in Hough space
Connection between image space (x, y) and Hough space (m, b)

A line in the image
corresponds to a point
in Hough space
A point in the image

corresponds to a line in
Hough space
Finding lines in Hough space
What is the line parameters for the line that contains both (x0 , y0 ) and
(x1 , y1 )?
The intersection of the lines b = –x0 m + y0 and b = –x1 m + y1 .
How to find the most likely parameters (m, b) for the most prominent line
in the image space?
Let each edge point in image space vote for a set of possible parameters in
Hough space
Accumulate votes in discrete set of bins; and parameters with the most votes
indicate line in image space.
Algorithm of Hough transform
Polar representation of lines
d: perpendicular distance from line to

origin
✓: angle the perpendicular makes with
the x-axis
x cos ✓ y sin ✓ = d
Basic algorithm
Initial H[d, ✓] = 0
For each edge point at [x, y] in the image:
for each ✓
d = x cos ✓ y sin ✓
H[d, ✓] = H[d, ✓] + 1
Find the value of (d, ✓) where H[d, ✓] is maximum
The line detected is given by d = x cos ✓ y sin ✓.
Example of Hough transform
Vote space and top peaks
Generalized to circle detection
Circle (x a)2 + (y b)2 = r2 : center (a, b) and radius r
Example
Summary of hough transform
Pros
All points are processed independently, so can cope with occlusion, gaps
Some robustness to noise: noise points unlikely to contribute consistently to
any single bin
Can detect multiple instances of a model in a single pass
Cons
Complexity of search time increases exponentially with the number of model
parameters
Non-target shapes can produce spurious peaks in parameter space
Quantization: can be tricky to pick a good grid size
Robust line detection is not easy
Robust line detection is not easy
Corner Detector
Flat region, edge and Corner
“flat” region: “edge”: “corner”:

no change as no change as shift significant change
shift window in window along the as shift window in
all directions edge direction all directions
10
Corner detection
A corner is easier to match, and edge detection fails at corner

right at corner, gradient is ill defined
Near corner, gradient has two different values.
Structure tensor (summing over a small region)
 P 2 P 
I PIx I2y = R 1 1 0
C= P x R,
Ix Iy Iy 0 2
where R is unitary matrix satisfying R> R = I.

Let x1 , x2 denote the eigenvectors respectively
Cx1 = 1 x1 , Cx2 = 2 x2 .
and suppose that 1 > 2

x1 = direction of largest increase in C
x2 = direction of smallest increase in C.
Classification of images points
λ2 “Edge”
λ2 >> λ1
“Corner”
λ1 and λ2 are large,
λ1 ~ λ2;
increases in all
directions
“Flat” region “Edge”

λ1 and λ2 are λ1 >> λ2
small;
λ1
Harris corner detector
Measurement of corner response
Harris Detector: Mathematics
R = det(C) k(trace(C))2 = 1 2 k( 1 + 2)
2
,
where k = 0.04 0.06.
λ2 “Edge” “Corner”
• R depends only on R<0
eigenvalues of M
• R is large for a corner R>0
• R is negative with large
magnitude for an edge
• |R| is small for a flat region
“Flat” “Edge”
|R| small R<0
λ1
21
Harris corner detector
Basic procedure
1 Compute the gradients at each point in the image
2 Compute C for each window to get its cornerness scores.
3 Compute the eigenvalues
4 Find points whose surrounding window gave large corner response
(f > threshold)
5 Take the points of local maxima, i.e., perform non-maximum suppression
Demonstration with different thresholds
yramid,
Scaleand then matching
invariant interestfeatures
points at the same level.
en does this work?
How can we independently select interest points in each image, such that
re efficient to extract features stable in both location and scale
the detections are repeatable across different scales?
extract
d scale that features
gives stable
local in both location
maxima and scale f in both position an
of a function
Find scale that gives local maxima of a function f in both position and scale.
K. Grauman]
Urtasun (TTI-C)
Ji Hui (National University of Singapore)
Computer Vision
Visual Information Interpretation: Image feature September 7, 2021
Jan 17, 201
29 / 34
Scale-invariant interesting point by DOG
Computing a set of differences of Gaussian filters (DOG) for finding 3D
(space+scale) maxima in the resulting structure.
Extreme within 3 ⇥ 3 ⇥ 3 regions in scale-space

Difference of Gaussian (DoG)
D( )f = (hk h ) ⌦ f.
Example
Local feature descriptor
Once keypoints are detected, the next question is how to describe them?
An ideal descriptor of keypoints
Robust to affine transform, lighting, noise
distinctive for accurate and fast matching
SIFT: Scale invariant Feature transform
Empirically found to show very good performance, invariant to image
rotation, scale, intensity change
Orientation histogram
SIFT is based on orientation histogram (weighted by magnitude)
Select canonical orientation

Assgin canonical orientation at peak of smoothed histogram, e.g., horizontal
on the histogram above.
Rotation invariance
Rotate all orientations by the dominant orientation
SIFT descriptor
4 ⇥ 4 Gradient window
Histogram of 4 ⇥ 4 samples per window in 8 directions
Gaussian weighting around center
4 ⇥ 4 ⇥ 8 = 128 dimensional feature vector

Part 07

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Part 07

Uploaded by

Copyright:

Available Formats

Visual Information Interpretation: Image feature

National University of Singapore

A feature is defined as an ”interesting” part of an image

Edges are caused by a variety of factors:

The gradient direction points to most rapid change in intensity

Differentiate a discrete image by finite difference operators

Edge point is located at the zero point

Compute edge magnitudes kr(G ⌦ I)k

Edge thinning by mathematical morphology operator.

original image norm of gradients

original Canny with Canny with

Edge is not line

Given points that belong to a line, what is the line?

Hough transform: a voting scheme to find lines

Connection between image space (x, y) and Hough space (m, b)

A point in the image

d: perpendicular distance from line to

Vote space and top peaks

“flat” region: “edge”: “corner”:

A corner is easier to match, and edge detection fails at corner

where R is unitary matrix satisfying R> R = I.

and suppose that 1 > 2

“Flat” region “Edge”

Extreme within 3 ⇥ 3 ⇥ 3 regions in scale-space

SIFT is based on orientation histogram (weighted by magnitude)

Select canonical orientation

You might also like