PillCam Foundation and Image Retrieval

Project Report
Georgia Institute of Technology
Medical Imaging
Spring 2007
Point of Contact
Todd Stokes
Instructor
Dr. May Wang
David
McCann
Stephen
Conover

Overview

Problem

Solution

Methodology

Implementation

Results

What is PillCam?

Wireless Capsule
Endoscopy (WCE)

Non-invasive

Low Complication Rate

Out Patient

Potentially End-to-End

Good diagnosis and
first look
http://www.nch.org/about/tech.shtml

Problem

Diagnosis Takes a Long Time

Where are we? What are we looking at?

Hours required to review video
10 ± 6 Time for physician to write report
10 ± 3 Time for nurse to begin download
30 ± 7 Time for nurse to prepare patients
100 ± 14 Time to read video by physician
Time (min) ± SD Task
* Bossa, et al. "Detection of Abnormal Lesions Recorded by Capsule Endoscopy: A Prospective Study
Comparing Endoscopist's and Nurse's Accuracy." Digestive and Liver Disease 38 (Aug 2006): 599-602.

Solution

Provide a tool to retrieve “similar” images
from a database

Allow reviewer to find related frames for
comparison/reference

Provide general 3D model as an
alternative to frames

Reduce review time through search

Methodology

Methodology Overview

Metrics

Symmetry

Curvature

Color

3D Modeling

Ranking

Methodology
Symmetry
Finding the “Center” of the Image

Rotational Symmetry

Adapted from Johansson et. al, “Detecting
Rotational Symmetries using
Normalized Convolution.”

Only looking for standard circular rotational
symmetry (no other basis functions).
Johansson, Bjorn; Knutsson, Hans; Granlund, Gosta. “Detecting Rotational Symmetries using Normalized
Convolution.” International Conference on Pattern Recognition (ICPR), vol 3: p. 3500.

Rotational Symmetry

Transform image
intensities into axial
and radial
components.

Try to find a center
point that minimizes
variance in one
transform and
maximizes it in the
other.

Nominal Results

Methodology
Curvature
Extracting the Shape of the Image

Curvature Overview
Why curvature? Good metric for features
and lumen/wall views.
A B C D

Process
1. Threshold
2. Extract Curvature
3. Histogram
4. Compare

Thresholding

Global average of intensity

Simple but effective

Sample results:
( )
( )
ij
ij
I Avg T
B G R I
·
+ + ·
3
1

Curvature Extraction

Curvature Equations (Osher and Sethien)
50 100 150 200 250
50
100
150
200
250
Binary Image Gaussian Kappa 1/Kappa
kappa
C
NormGrad NormGrad
w kappa
normGrad normGrad
w
NormGrad
Y
Y
X
X ij
ij
Y X
1
elsewhere 0
) max( * 75 . 0 where 1
2 2
2 2
·

,
`

.
| ∇
∇ +
,
`

.
| ∇
∇ ·
¹
'
¹ >
·
∇ + ∇ ·
φ φ
φ φ

Base Test Case
0 50 100 150 200
0
50
100
150
Curvature
C
o
u
n
t
Curvature Histogram
 
 
50 100 150 200 250
50
100
150
200
250
 
 
50 100 150 200 250
50
100
150
200
250
30
40
50
60
70
80
90
100
Circle
Radius of 50
Extracted
Curvature
Histogram of
Curvature is Centered
Around 50!

Process Development and Refinement I

Initial draft in MATLAB m-code

Ported to and refined in Java

Java Application for Testing:

Process Development and Refinement II
Bad
Better
Best
Dilated
& Masked
Masked

Histogram

2D Curvature Reduced 1D Array

Reduced to 100 bin histogram

Count is divided by # pixels for scaling
0 50 100 150 200
0
50
100
150
Curvature
C
o
u
n
t
Curvature Histogram

Comparison

Sum of the squares difference in
histogramed values 0:10:1000:

Must be normalized afterwards:
( )

·
− ·
N
n
B A
n Bin n Bin
1
2
] [ ] [ Score Comparison
( ) ) ( Score
Min(Score) - Score
Score Comparison
Normalized
Score Min Max −
·

Methodology
Color

Overview

HSI Histogram

Sum Power Compare

Methodology
Final Scoring

·
·
Metrics
k
i i
s w
#
1
Score Final

Methodology
3D Modeling

Overview

3D OBJ Model interpolated from MRI slices

Basic lighting and flat color texturing

Smoothed Surface Normals

The surface normal at
a given vertex is the
average of the
surface normals of the
polygons meeting at
the vertex.

Methodology
Summary
Final Rationale: How does all this work together?

Implementation

Implementation Overview

Completely Java

OpenGL and JOGL

30 files

3,000 SLOC

Class Diagram - Model

Modules
Curvature
Symmetry
Color
Sum of
Squares
Distance
Score All
Present
Results
Single
Image Load
3D Model
Navigation
Tracks
Rendering
N
o
r
m
a
l
i
z
e
Source
Selection
Metric
Extraction
Comparison Results

Features

Easy to use graphical user interface

Navigate manually through model with AWSD and
mouse

Record camera positions to a track

Save and load camera tracks

Quickly scan through loaded camera track

Add images and directories of images to database

Compare database images to 3D model or image file

Quick link back to results

Save/Load feature database functionality

Weaknesses

Thresholding Simple

Simple Ranking

Need higher fidelity model

Slow feature extraction

Results

Interface

Results
Video
Frame
Independent
Image
Model
Frame
Bad Frame
Bad Results

Results I: Search for Image from Video
1 2
3 4

Results II: Search for Image from Independent Frame
1 2
3 4

Results III: Search for Image from 3D Model
1 2
3 4
5

Results IV: Bad Results
1 2
3 4

Summary

Metrics perform well

Need deformable model

Review time is improved

Questions?

Master your semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master your semester with Scribd & The New York Times

Cancel anytime.