You are on page 1of 42

Georgia Institute of Technology

Medical Imaging
Spring 2007

Instructor
Dr. May Wang
Point of Contact
Todd Stokes

PillCam Foundation and Image Retrieval

Project Report
Stephen David
Conover McCann
Overview
 Problem
 Solution
 Methodology
 Implementation
 Results
What is PillCam?
 Wireless Capsule
Endoscopy (WCE)
 Non-invasive
 Low Complication Rate
 Out Patient
 Potentially End-to-End
 Good diagnosis and
first look
http://www.nch.org/about/tech.shtml
Problem
 Diagnosis Takes a Long Time
 Where are we? What are we looking at?
 Hours required to review video

Task Time (min) ± SD


Time to read video by physician 100 ± 14
Time for nurse to prepare patients 30 ± 7
Time for nurse to begin download 10 ± 3
Time for physician to write report 10 ± 6
* Bossa, et al. "Detection of Abnormal Lesions Recorded by Capsule Endoscopy: A Prospective Study
Comparing Endoscopist's and Nurse's Accuracy." Digestive and Liver Disease 38 (Aug 2006): 599-602.
Solution
 Provide a tool to retrieve “similar” images
from a database
 Allow reviewer to find related frames for
comparison/reference
 Provide general 3D model as an
alternative to frames
 Reduce review time through search
Methodology
Methodology Overview
 Metrics
 Symmetry
 Curvature
 Color

 3D Modeling
 Ranking
Methodology

Symmetry
Finding the “Center” of the Image
Rotational Symmetry
 Adapted from Johansson et. al, “Detecting
Rotational Symmetries using
Normalized Convolution.”
 Only looking for standard circular rotational
symmetry (no other basis functions).

Johansson, Bjorn; Knutsson, Hans; Granlund, Gosta. “Detecting Rotational Symmetries using Normalized
Convolution.” International Conference on Pattern Recognition (ICPR), vol 3: p. 3500.
Rotational Symmetry
 Transform image
intensities into axial
and radial
components.
 Try to find a center
point that minimizes
variance in one
transform and
maximizes it in the
other.
Nominal Results
Methodology

Curvature
Extracting the Shape of the Image
Curvature Overview

Why curvature? Good metric for features


and lumen/wall views.

A B C D
Process
1. Threshold
2. Extract Curvature
3. Histogram
4. Compare
Thresholding
 Global average of intensity
 Simple but effective
 Sample results:

1
I ij = ( R + G + B )
3
T = Avg ( I ij )
Curvature Extraction
 Curvature Equations (Osher and Sethien)
NormGrad = ∇ X φ 2 + ∇Y φ 2
1 where normGrad > 0.75 * max(normGrad )
wij = 
 0 elsewhere
2 2
 ∇ Xφ   ∇Y φ 
kappa = wij ∇X   + ∇Y  
 NormGrad   NormGrad 
1
C =
kappa

Binary Image Gaussian Kappa 1/Kappa

50

100

150

200

250
50 100 150 200 250
Base Test Case
Curvature Histogram
  100
 

90
50 150
50
80

100 100 70
100

Count
60
150 150
50

50
200 200 40

30
250  
250
50 100 150 200 250   0
50 100 150 200 250 0 50 100 150 200
Curvature

Circle Extracted Histogram of


Radius of 50 Curvature Curvature is Centered
Around 50!
Process Development and Refinement I

 Initial draft in MATLAB m-code


 Ported to and refined in Java
 Java Application for Testing:
Process Development and Refinement II

Bad

Better
Masked

Best
Dilated
& Masked
Histogram
 2D Curvature Reduced 1D Array
 Reduced to 100 bin histogram
 Count is divided by # pixels for scaling

Curvature Histogram

150

100
Count

50

0
0 50 100 150 200
Curvature
Comparison
 Sum of the squares difference in
histogramed values 0:10:1000:
N
Comparison Score = ∑ ( Bin A [n] − BinB [n])
2

n =1

 Must be normalized afterwards:

Score - Min(Score)
Comparison Score Normalized =
Max( Score ) − Min( Score)
Methodology

Color
Overview
 HSI Histogram
 Sum Power Compare
Methodology # Metrics

Final Scoring Final Score = ∑w s


k =1
i i
Methodology

3D Modeling
Overview
 3D OBJ Model interpolated from MRI slices
 Basic lighting and flat color texturing
Smoothed Surface Normals
 The surface normal at
a given vertex is the
average of the
surface normals of the
polygons meeting at
the vertex.
Methodology

Summary
Final Rationale: How does all this work together?
Implementation
Implementation Overview
 Completely Java
 OpenGL and JOGL
 30 files
 3,000 SLOC
Class Diagram - Model
Modules

Single Color
Sum of

Normalize
Image Load
Squares
Curvature Present
Score All
Results
3D Model
Navigation Symmetry Distance
Tracks
Rendering

Source Metric Comparison Results


Selection Extraction
Features
 Easy to use graphical user interface
 Navigate manually through model with AWSD and
mouse
 Record camera positions to a track
 Save and load camera tracks
 Quickly scan through loaded camera track
 Add images and directories of images to database
 Compare database images to 3D model or image file
 Quick link back to results
 Save/Load feature database functionality
Weaknesses
 Thresholding Simple
 Simple Ranking
 Need higher fidelity model
 Slow feature extraction
Results
Interface
Results
Video Independent Model Bad Frame
Frame Image Frame Bad Results
Results I: Search for Image from Video

1 2

3 4
Results II: Search for Image from Independent Frame

1 2

3 4
Results III: Search for Image from 3D Model

1 2

3 4

5
Results IV: Bad Results

1 2

3 4
Summary
 Metrics perform well
 Need deformable model
 Review time is improved
 Questions?

You might also like