EE 475 Digital Image Processing Term Project

Interactive Segmentation Using

Intelligent Scissors
Instuctor: Professor Bulent Sankur Prepeared by: Yusuf Ziya Isik and Ashat Turlibayev

Intelligent Scissors
Fully automatic segmentation is an unsolved problem due to wide variety of images. Intelligent Scissors is a semi-automatic general purpose segmentation tool.

The efficient and accurate boundary extraction, which requires minimal user input with a mouse, is obtained. The underlying mechanism for the Intelligent Scissors is the live-wire path selection tool.

Live wire tool

Boundary detection is formulated as an optimal path search in a weighted graph. Optimal boundaries are computed as the user moves the mouse starting from a manually specified seed point.

Live-wire segmentation
When the mouse position comes in proximity to an object edge, a live-wire boundary snaps to, and wraps around the object of interest. Input of a new seed point freezes the selected boundary segment, and the process is repeated until the boundary is complete.

Image as a Weighted Graph

Basic idea is : Formulate image as a weighted graph where pixels represent nodes with directed, weighted edges connecting them to neighbors.

LIVE WIRE BOUNDARY SNAPPING

The boundary finding consists of finding the globally optimal path from the start node to a goal node. The optimal path or boundary is defined as the minimum cumulative cost path from a start node to a goal node where the cumulative cost of a path is the sum of the local costs on the path.This is done by the efficient use of Dijkstras graph searching algorithm.

Local Costs
Since an optimal path corresponds to a segment of an object boundary, pixels that exhibit strong edge features are made to have low local costs. The local cost function is a weighted sum of the component cost functions for each of the following features: i) the laplacian zero-crossing, fZ, ii) the gradient magnitude, fG and iii) the gradient direction, fD.

Local Costs
Letting l(p,q) represent the local cost for the directed link from pixel p to a neighboring pixel q l(p,q) =wGf G (q) + wZf Z (q) + wDf D (p,q)

The laplacian zero-crossing feature, fZ(q)

it is a binary edge feature used for boundary localization. if IL is the output of the convolution of the image with a laplacian edge operator

The laplacian zero-crossing feature, fZ(q) However,a discrete Laplacian image produces very few actual zero values. Therefore, if IL(q) has a neighbor with a different sign, of the two pixels, the one closest to zero represents the zero-crossing or the position to localize the boundary.

The gradient magnitude feature, fG(q)

It is a strong measure of edge strength, and is heavily weighted. High image gradients will correspond to low costs, therefore:

The gradient direction.

It adds a smoothness constraint to the boundary by associating a high cost for sharp changes in boundary direction. The gradient direction is the unit vector defined by the image gradients, Gx and Gy in the x and y direction, respectively.

The formulation of the gradient direction feature cost

Boundary Detection as Graph Searching

The graph search algorithm is initialized by placing a start or seed point, s, with a cumulative cost of 0, on an otherwise empty active list, L All the other points in the image are initialized with infinite cumulative costs.

Boundary Detection as Graph Searching

A point p is placed on the active list in sorted order based on its total or cumulative cost, g(p). In each iteration, the point or pixel p with minimum cumulative cost is removed from L and expanded by computing the total cost to each of ps unexpanded neighbors

For each neighbor q of p, the cumulative cost to q :gtmp = g(p)+l(p,q). If gtmp< g(q) then g(q) is assigned the new cumulative cost and an optimal path pointer is set from q back to p. The process repeats until al the image pixels have been expanded.

The cost expansion algorithm

The cost expansion algorithm

Interactive "Live-Wire" Segmentation Tool

Once the optimal path pointers are generated, a desired boundary segment can be chosen dynamically via the free point specified by the current cursor position. Interactive movement of the free point by the mouse cursor causes the boundary to behave like a live wire as it erases the previous boundary points and displays the new minimum cost path defined by following path pointers from the free point back to the seed point.

By constraining the seed point and free points to lie near a given edge, the user is able to interactively "snap" and "wrap" the live-wire boundary around the object of interest.

new seed points

When movement of the free point causes the boundary to digress from the desired object edge, input of a new seed point prior to the point of departure effectively "ties off" or freezes the boundary computed up to the new seed point and reinitiates the boundary detection starting from the new seed point.

new seed points

Interactive "Live-Wire" Segmentation Tool

Since each pixel (or free point) defines only one optimal path to a seed point, a minimum of two seed points must be deposited to ensure a closed object boundary. Input of a new seed point freezes the selected boundary segment, and the process is repeated until the boundary is complete.

A minimum of two points are required

A cursor snap
Placing seed points directly on an object's edge is often difficult and tedious. To facilitate seed point placement, a cursor snap is available which forces the mouse pointer to the maximum gradient magnitude pixel within a user specified neighborhood. The neighborhood can vary from 1x1 (resulting in no cursor snap) to 15x15 (where the cursor can snap as much as 7 pixels in both x and y).

Two novel enhancements:

Two novel enhancements to the basic live wire methodology include boundary cooling and on-the-fly training. Data-driven boundary cooling generates seed points automatically and further reduces user input. On-the-fly training adapts the dynamic boundary to edges of current interest.

Results:

Results:

CONCLUSIONS

Live-wire boundary extraction provides an accurate and efficient interactive tool for image segmentation. The live-wire tool is intuitive to use and can be applied to black and white or color images of arbitrary content and complexity. As a practical matter, live-wire boundary snapping typically requires less time and effort to accurately segment an object than it takes to manually input an initial approximation to the object boundary.