Professional Documents
Culture Documents
PY
Giger CIBC Lecture 2019 Giger CIBC Lecture 2019
O
Recent & Current Graduate Students
CAD, Radiomics, and AI in Breast Imaging
Joel Wilkie, PhD
Martin King, PhD
Nick Gruszauskas, PhD
Yading Yuan, PhD
Robert Tomek, MS
Neha Bhooshan, PhD
Andrew Jamieson, PhD
Hsien-Chi Kuo, PhD
Giger Lab
Research Lab
Karen Drukker, PhD
C • The focus is the quantitative image analysis of images “clinically &
routinely” obtained on the population.
• We want to ask questions about the relationships between features
“seen” in medical images and the biology of cancer so that eventually
we can detect/diagnose disease early and give the right patient the
Martin Andrews, PhD Hui Li, PhD Over the past 3 decades-- At right treatment at the right time.
T
William Weiss, PhD Heather Whitney, PhD the University of Chicago, we
Chris Haddad, PhD Yu Ji, MD discover new ways to use
Natasha Antropova, PhD Chun Wai Chan, MS computers (AI) to enrich the
Adam Sibley, PhD Li Lan, MS information extracted from
Kayla Robinson, PhD John Papaioannou, MS medical images so that
O
PY
Giger CIBC Lecture 2019 Giger CIBC Lecture 2019
O
Deep Learning C Deep Learning
conv
conv
conv
conv
conv
maxconv
conv
maxconv
pool
pool
max
max
conv
conv
conv
conv
conv
conv
pool
pool
conv
conv
conv
conv
conv
conv
pool
pool
max
max
fc
fc
fc
fc
conv
conv
conv
conv
conv
conv
pool
pool
max
max
pool
pool
max
max
fc
fc
Probability Probability
T
of disease of disease
O
In deep learning, each level learns to transform its input data Higher-level concepts are defined in terms of lower-level ones,
into a slightly more abstract and composite representation. the hierarchy of such features is called a deep architecture.
Giger CIBC Lecture 2019 Giger CIBC Lecture 2019
N
O
• Transfer Learning
conv
conv
conv
pool
conv
conv
conv
pool
max
pool
max
fc
Developing Density
CAD, Radiomics, and AI in Breast Imaging
PY
Giger CIBC Lecture 2019 Mammography (x-ray) Giger CIBC Lecture 2019
O
AI/Deep learning example in CADe
University of Chicago
1994 Prototype System for Computer-Aided Detection
• FDA-approved
concurrent-read AI • FDA-cleared concurrent-
system (QView Medical) read AI for breast
for 3D ultrasound tomosynthesis (iCAD
• Reduces ABUS exam PowerLook)
interpretation time • Improves accuracy and
while maintaining
efficiency
diagnostic accuracy
• Uses deep learning • Uses deep learning
PY
Giger CIBC Lecture 2019 Giger CIBC Lecture 2019
O
Computer-aided diagnosis in the work-up of suspect lesions:
CAD, Radiomics, and AI in Breast Imaging
•
•
•
What is CAD, Radiomics, & AI?
Role in breast cancer detection
Role in breast cancer diagnosis
C malignant vs. benign lesions
• Use computer output to help characterize (i.e., output descriptors of
the lesion) and potentially indicate a computer-determined probability
of malignancy of a found lesion
• The final decision on patient management is still made by the
radiologist
T
• Role in breast cancer therapy
• Challenges
• Future
O
CNN Schematic
PY
Huynh B, Li H, Giger ML: Digital mammographic tumor classification using transfer learning from deep convolutional neural
networks. J Medical Imaging 3(3), 034501 (2016). Giger CIBC Lecture 2019 Giger CIBC Lecture 2019
O
Quantitative radiomics in distinguishing between malignant Computer-extracted Breast Cancer on MRI
and benign breast lesions
Human-Engineered Radiomics Convolutional Neural Networks
(CNN) Radiomics
C (can analyze as a “virtual” digital biopsy of the tumor)
• automatic
segmentation &
analysis
T
Computerized Tumor Segmentation
• 3D
Computer-Extracted Tumor Features • non-invasive
• covers complete
CNN tumor
O
Schematic
• repeatable
Classification on clinical question
Huynh B, Li H, Giger ML: Digital mammographic tumor classification using transfer learning from
deep convolutional neural networks. J Medical Imaging 3(3), 034501 (2016).
Giger CIBC Lecture 2019 Giger CIBC Lecture 2019
N
O
Computer-Extracted Tumor Features • CNNs extract features from entire ROIs without localization or
CNN Schematic
segmentation of lesions.
• Advantage: No lesion segmentation is required
Classification on clinical question
• Advantage: No extraction of segmentation-based features, such as size,
shape, margin sharpness, texture, and kinetics
PY
Huynh B, Li H, Giger ML: Digital mammographic tumor classification using transfer learning from deep convolutional neural
networks. J Medical Imaging 3(3), 034501 (2016). Giger CIBC Lecture 2019
O
Conventional CAD/Radiomics & Deep Learning CAD/Radiomics Conventional CADx & Deep Learning CADx
Likelihood of
(task of distinguishing between cancers and non cancers)
RED = CANCER
C (diagnostic task of distinguishing between cancers and non
cancers across breast imaging modalities; ROC analysis)
Breast Imaging
Modality
Number of
Cases
Human-
Engineered CADx
(AUC)
Deep
Learning CNN
(AUC)
Combination
Human-Engineered
& CNN
being cancer as GREEN = Non- (AUC)
T
determined from CANCER Digital 245 0.79 0.81 0.86
human- Mammography
engineered
Ultrasound 1125 0.84 0.87 0.90
CADx
DCE-MRI 690 0.86 0.87 0.89
O
Antropova N, Huynh BQ, Giger ML: A deep fusion methodology for breast cancer diagnosis demonstrated on three
imaging modality datasets. Medical Physics online doi.org/10.1002/mp.12453, 2017.
Likelihood of being cancer as
determined from deep learning Giger CIBC Lecture 2019 Giger CIBC Lecture 2019
N
O
Effect of Three MRI Input Protocols on CNN Effect of Three MRI Input Protocols on CNN CADx
D
Central slice of the 2nd post- Central slice of the 2nd post- Maximum intensity Central slice of the 2nd post- Central slice of the 2nd post- Maximum intensity
contrast MRI contrast subtraction MRI projection (MIP) contrast MRI contrast subtraction MRI projection (MIP)
2-D information 2-D information 2-D information 2-D information 2-D information 2-D information
Temporal information Temporal Temporal information Temporal
Antropova N, Abe H, Giger ML: Use of clinical MRI
information information
Antropova N, Abe H, Giger ML: Use of clinical
maximum intensity projections for improved breast Volume information MRI maximum intensity projections for Volume information
lesion classification with deep CNNs J Med Imaging 2018 improved breast lesion classification with deep
Jan;5(1):014503. Giger CIBC Lecture 2019 CNNs J Med Imaging 2018 Jan;5(1):014503. Giger CIBC Lecture 2019
Radiomics/Machine Learning
AnalysisWorkstation
& Outputforofthe High Throughput
Tumor Signature MRI Phenotyping of CADx for Breast MRI
Breast Lesions – DIAGNOSTIC TASKS
Automated Lesion Segmentation, Feature Extraction [volumetrics, morphological, texture, kinetics] and Estimation of the
Probability of Malignancy
Quantitative
Insights is now
PY
Giger CIBC Lecture 2019
(de novo clearance)
Giger et al., RSNA 2010
O
CADx for Breast Ultrasound
• Koios DS (Decision Support) Breast 2.0 is intended for use to
assist physicians analyzing breast ultrasound images.
• Aligns a machine learning-generated probability of malignancy
C CAD, Radiomics, and AI in Breast Imaging
•
•
•
What is CAD, Radiomics, & AI?
Role in breast cancer detection
Role in breast cancer diagnosis
with the appropriate BI-RADS category.
T
• Role in breast cancer therapy
• 510(k) clearance • Challenges
• Future
O
Predicting recurrence-free survival “early on” in neoadjuvant treatment of Predicting recurrence-free survival “early on” in neoadjuvant treatment of
D
breast cancer (most-enhancing tumor volume by MRI radiomics) breast cancer (most-enhancing tumor volume by MRI radiomics)
A subset, based on availability, of the ACRIN 6657
dynamic contrast-enhanced MR images was used
in which we analyzed images of all women
imaged at
• pre-treatment baseline (141 women: 40 with a
recurrence, 101 without) and
• all those imaged after completion of the first
cycle of chemotherapy, i.e., at early treatment Kaplan-Meier recurrence-free survival estimates for METV at the early treatment time
(143 women: 37 with a recurrence vs. 105 point using the highest quartile cut-point (Q3) with corresponding p-values by hormone-
without). receptor status subgroup: hormone-receptor positive and HER2 negative (N=66, left), HER2
Drukker K, Li H, Antropova N, Edwards A, Papaioannou J, positive (N=38, middle), and triple negative (N=36, right) with corresponding p-values (for 2
Giger ML: Most-enhancing tumor volume by MRI
radiomics predicts recurrence-free survival “early on”
cases the hormone receptor status was unknown)
in neoadjuvant treatment of breast cancer. Cancer Drukker K, Li H, Antropova N, Edwards A, Papaioannou J, Giger ML: Most-enhancing tumor volume by MRI radiomics predicts
Imaging 18:12, 2018 Giger CIBC Lecture 2019 Giger CIBC Lecture 2019
recurrence-free survival “early on” in neoadjuvant treatment of breast cancer. Cancer Imaging 18:12, 2018
PAM50 ROR-P
recurrence)
0.96
recurrence)
53.2 data.
(Subtype+Proliferation) (low risk of breast cancer (high risk of breast cancer
Range [-13.21, 72.38] recurrence) recurrence)
Radiomics for MRI Tumor Size
“virtual” biopsy
(Effective Diameter)
Range [7.8 54.0]
16.8 mm 21.7 mm
• Application stage – developing predictive models for use in risk
MRI Tumor Irregularity
Range [0.40 0.84]
MRI Tumor
0.438 0.592 assessment, screening, detection, diagnosis, prognosis,
Heterogeneity (Entropy) 6.27 6.51
Range [6.00 6.59] therapeutic response, risk of recurrence, etc.
Li H, Zhu Y, Burnside ES, …. Perou CM, Ji Y*, Giger ML*: MRI radiomics signatures for predicting the risk of breast cancer
PY
recurrence as given by research versions of gene assays of MammaPrint, Oncotype DX, and PAM50. Radiology DOI: Giger CIBC Lecture 2019
http://dx.doi.org/10.1148/radiol.2016152110, 2016. Giger CIBC Lecture 2019
O
Two Stage Process: Discovery and Predictive Modeling for
How do we Harness this Imaging Big Data?
Diagnostic
Imaging
Personalized Patient Care
IMAGING-GENOMICS
Biopsy Results,
Genetic Testing
DISCOVERY
Virtual “digital”
biopsies
TRANSLATION:
Predictive
and clinical data, molecular data, genomic data, and outcome Results Modeling
T
data.
Treatment Planning
& Following for Response
• Application stage – developing predictive models for use in risk Virtual “digital”
biopsies
O
Exploratory Cluster Analysis of the MRI Tumor IMAGING GENOMICS – USING VIRTUAL BIOPSIES Transcriptional activities of various
D
• Guo W, Li H, Zhu Y, …, Giger ML*, Ji Y*: Prediction of clinical phenotypes in invasive breast carcinomas from the integration of
radiomics and genomics data. J Medical Imaging 2(4), 041007 (Oct-Dec 2015).
• Zhu Y, Li H, … Giger ML*, Ji Y*: Deciphering genomic underpinnings of quantitative MRI-based radiomic phenotypes of
Zhu Y, Li H, … Giger ML*, Ji Y*: Deciphering genomic underpinnings Giger
of quantitative
CIBC LectureMRI-based radiomic phenotypes of invasive breast carcinoma. Nature
invasive breast carcinoma. Nature – Scientific ReportsGiger
5:17787 (2015)
CIBC Lecture 2019
– Scientific Reports 5:17787 (2015)
2019
If the sample does not accurately represent the population of interest, statistics
are not meaningful.
PY
Giger CIBC Lecture 2019 Giger CIBC Lecture 2019
O
Robustness: Radiomics Robustness: Radiomics
Differences in image acquisition or the population may affect computer-extracted
image-based phenotypes
• Manufacturer
• Imaging protocol
• Geographic location
• Spatial resolution
• Noise
• System gain
C Differences in image acquisition or the population may affect computer-extracted
image-based phenotypes
• Manufacturer
• Imaging protocol
• Geographic location
• Disease prevalence
• Geographical
• Single vs. multiple institutions
• Lack of harmonization
• Racial differences in disease prevalence and characteristics • Racial differences in disease prevalence and characteristics
T
• Actual outcome data such as survival may not be available and intermediate • Actual outcome data such as survival may not be available and intermediate
alternatives may need to be used alternatives may need to be used
Big data Medium data Small data Big data Medium data Small data
Standard of care Clinical trials Pilot studies Standard of care Clinical trials Pilot studies
O
If the sample does not accurately represent the population of interest, statistics If the sample does not accurately represent the population of interest, statistics
are not meaningful. are not meaningful.
Giger CIBC Lecture 2019 Giger CIBC Lecture 2019
N
O
If the sample does not accurately represent the population of interest, statistics
are not meaningful.
Giger CIBC Lecture 2019 Giger CIBC Lecture 2019
PY
Giger CIBC Lecture 2019
O
Thank you to Giger Lab
Recent & Current Graduate Students
Joel Wilkie, PhD
Martin King, PhD
Nick Gruszauskas, PhD
Yading Yuan, PhD
Robert Tomek, MS
Neha Bhooshan, PhD
Andrew Jamieson, PhD
Research Lab
Karen Drukker, PhD
C
Collaborators
Hsien-Chi Kuo, PhD Hui Li, PhD
T
Gillian Newstead, MD
Martin Andrews, PhD Heather Whitney, PhD Suzanne Conzen, MD
William Weiss, PhD Yu Ji, MD
Marcus Clark, MD
Chris Haddad, PhD Chun Wai Chan, MS
Li Lan, MS Yuan Ji, PhD
Natasha Antropova, PhD
John Papaioannou, MS Greg Karczmar, PhD
Adam Sibley, PhD
Milica Medved, PhD
O