PillCam Foundation and Image Retrieval

Project Report
Georgia Institute of Technology
Medical Imaging
Spring 2007
Point of Contact
Todd Stokes
Instructor
Dr. May Wang
David
McCann
Stephen
Conover

Overview

Problem

Solution

Methodology

Implementation

Results

What is PillCam?

Wireless Capsule
Endoscopy (WCE)

Non-invasive

Low Complication Rate

Out Patient

Potentially End-to-End

Good diagnosis and
first look
http://www.nch.org/about/tech.shtml

Problem

Diagnosis Takes a Long Time

Where are we? What are we looking at?

Hours required to review video
10 ± 6 Time for physician to write report
10 ± 3 Time for nurse to begin download
30 ± 7 Time for nurse to prepare patients
100 ± 14 Time to read video by physician
Time (min) ± SD Task
* Bossa, et al. "Detection of Abnormal Lesions Recorded by Capsule Endoscopy: A Prospective Study
Comparing Endoscopist's and Nurse's Accuracy." Digestive and Liver Disease 38 (Aug 2006): 599-602.

Solution

Provide a tool to retrieve “similar” images
from a database

Allow reviewer to find related frames for
comparison/reference

Provide general 3D model as an
alternative to frames

Reduce review time through search

Methodology

Methodology Overview

Metrics

Symmetry

Curvature

Color

3D Modeling

Ranking

Methodology
Symmetry
Finding the “Center” of the Image

Rotational Symmetry

Adapted from Johansson et. al, “Detecting
Rotational Symmetries using
Normalized Convolution.”

Only looking for standard circular rotational
symmetry (no other basis functions).
Johansson, Bjorn; Knutsson, Hans; Granlund, Gosta. “Detecting Rotational Symmetries using Normalized
Convolution.” International Conference on Pattern Recognition (ICPR), vol 3: p. 3500.

Rotational Symmetry

Transform image
intensities into axial
and radial
components.

Try to find a center
point that minimizes
variance in one
transform and
maximizes it in the
other.

Nominal Results

Methodology
Curvature
Extracting the Shape of the Image

Curvature Overview
Why curvature? Good metric for features
and lumen/wall views.
A B C D

Process
1. Threshold
2. Extract Curvature
3. Histogram
4. Compare

Thresholding

Global average of intensity

Simple but effective

Sample results:
( )
( )
ij
ij
I Avg T
B G R I
·
+ + ·
3
1

Curvature Extraction

Curvature Equations (Osher and Sethien)
50 100 150 200 250
50
100
150
200
250
Binary Image Gaussian Kappa 1/Kappa
kappa
C
NormGrad NormGrad
w kappa
normGrad normGrad
w
NormGrad
Y
Y
X
X ij
ij
Y X
1
elsewhere 0
) max( * 75 . 0 where 1
2 2
2 2
·

,
`

.
| ∇
∇ +
,
`

.
| ∇
∇ ·
¹
'
¹ >
·
∇ + ∇ ·
φ φ
φ φ

Base Test Case
0 50 100 150 200
0
50
100
150
Curvature
C
o
u
n
t
Curvature Histogram
 
 
50 100 150 200 250
50
100
150
200
250
 
 
50 100 150 200 250
50
100
150
200
250
30
40
50
60
70
80
90
100
Circle
Radius of 50
Extracted
Curvature
Histogram of
Curvature is Centered
Around 50!

Process Development and Refinement I

Initial draft in MATLAB m-code

Ported to and refined in Java

Java Application for Testing:

Process Development and Refinement II
Bad
Better
Best
Dilated
& Masked
Masked

Histogram

2D Curvature Reduced 1D Array

Reduced to 100 bin histogram

Count is divided by # pixels for scaling
0 50 100 150 200
0
50
100
150
Curvature
C
o
u
n
t
Curvature Histogram

Comparison

Sum of the squares difference in
histogramed values 0:10:1000:

Must be normalized afterwards:
( )
∑
·
− ·
N
n
B A
n Bin n Bin
1
2
] [ ] [ Score Comparison
( ) ) ( Score
Min(Score) - Score
Score Comparison
Normalized
Score Min Max −
·

Methodology
Color

Overview

HSI Histogram

Sum Power Compare

Methodology
Final Scoring
∑
·
·
Metrics
k
i i
s w
#
1
Score Final

Methodology
3D Modeling

Overview

3D OBJ Model interpolated from MRI slices

Basic lighting and flat color texturing

Smoothed Surface Normals

The surface normal at
a given vertex is the
average of the
surface normals of the
polygons meeting at
the vertex.

Methodology
Summary
Final Rationale: How does all this work together?

Implementation

Implementation Overview

Completely Java

OpenGL and JOGL

30 files

3,000 SLOC

Class Diagram - Model

Modules
Curvature
Symmetry
Color
Sum of
Squares
Distance
Score All
Present
Results
Single
Image Load
3D Model
Navigation
Tracks
Rendering
N
o
r
m
a
l
i
z
e
Source
Selection
Metric
Extraction
Comparison Results

Features

Easy to use graphical user interface

Navigate manually through model with AWSD and
mouse

Record camera positions to a track

Save and load camera tracks

Quickly scan through loaded camera track

Add images and directories of images to database

Compare database images to 3D model or image file

Quick link back to results

Save/Load feature database functionality

Weaknesses

Thresholding Simple

Simple Ranking

Need higher fidelity model

Slow feature extraction

Results

Interface

Results
Video
Frame
Independent
Image
Model
Frame
Bad Frame
Bad Results

Results I: Search for Image from Video
1 2
3 4

Results II: Search for Image from Independent Frame
1 2
3 4

Results III: Search for Image from 3D Model
1 2
3 4
5

Results IV: Bad Results
1 2
3 4

Summary

Metrics perform well

Need deformable model

Review time is improved

Questions?