1 / 29

Problem Statement:

●
Input : Image I (resolution: n x m)
new size (resolution : n’ x m’)

●
Output: Image I’ of size n’ x m’ which will be
“good representative” of the original
image.

2 / 29
 Suggested Solutions:

●
Cropping
Drawing a rectangular box and extracting the
desired portion of the image.

●
Scaling
Stretching/Squeezing image as needed so it
fits a desired size.

3 / 29
4 / 29
 Problems with suggested solutions:
●
Cropping:

5 / 29
Scaling

6 / 29
 Our solution to resizing problem:

●
Remove unimportant pixels than blend with
their background.
●
Unimportant : ‘pixels with less energy’
●
Human vision more sensitive to edges – so
try remove content from smoother areas

7 / 29
Seam Carving:

8 / 29
 Seam:

●
A connected path of pixels from top to
bottom (or left to right).

●
Exactly one in each row (or column)

●
Classification of Seam:
1. Vertical Seam
2. Horizontal Seam

9 / 29
Vertical Seam:

Original Image Vertical Seams 10 / 29

Horizontal Seam:

Original image Horizontal Seams

11 / 29
Conversion to Energy format:

Original Image Energy format

12 / 29
 Optimal Seam

●
The optimal seam is one with lowest energy
in the image.
●
The lowest-energy seam is the one whose
total energy across all the pixels in the seam
is minimized.
●
Its an optimization problem.

13 / 29
 Why Greedy approach will fail ?
●
The lowest-energy seam may not pass
through all the lowest-energy pixels in the
image.
●
It is the total energy of the seam being
minimized, not the individual pixel energies.

14 / 29
 Dynamic Programming to the rescue:

●
Recurrence relation for finding Cumulative
Minimum Energy Matrix:

M(x,y) : Minimal cost of a seam going

through (x,y)
e(x,y) : Energy at pixel (x,y)

15 / 29
 While Computing Vertical Seam:

●
PS: Similarly, can be done for Horizontal 16 / 29
seam.
Sample Dry Run:

17 / 29
Backtracking the seam

18 / 29
 What happens to overall energy in the
image during Seam carving ?
●
During seam carving we remove as many
low energy pixels and as few high energy
pixels.
●
As a result the average energy increases.
●
The inserted energy is due to new edges
created by previously non adjacent pixels
that become neighbors once the seam is
removed.
●
But, the average energy has to be preserved.

19 / 29
 Optimization:

Minimize Inserted Energy

●
Instead of removing the seam of least
energy, remove the seam that inserts the
least energy to the image

●
The idea is to look forward while removing
the seams. (Forward Energy)

20 / 29
Forward Energy:

21 / 29
22 / 29
Forward vs. Backward

23 / 29
Object Removal:

24 / 29
Pigeon removed Girl removed
●
In object removal, we are attracting the
seams to pass through the object.

●
This can we done by adding negative weights
to the pixels of the object and then running
the seam carving algorithm.

25 / 29
●
To preserve a certain portion of the image,
add higher weights to the corresponding
pixels in the energy matrix.

●
This ensures that the seam does not pass
through those pixels.

26 / 29
 Failure Cases:
●
With images that have many long straight
lines

27 / 29
●
High frequency background

28 / 29
THANK YOU !

29 / 29