Medical Imaging using

Deep Learning for

Deep Learning

-by
Mr. Amar Saraswat
PhD – Pursuing
NOIDA INTERNATIONAL UNIVERSITY
 Agenda

 What is Deep Learning?

 Deep Learning models in Medical Imaging
 Rise of Specialized Compute
 Techniques for Optimization
 E2E Pipeline
 Look into future
 References
 What is
Deep
Learning?

 Learn like humans

 Automated Feature Learning

 Layers are like Image

Filters

 AI Neural Networks composed of

Many layers
 Deep
Learning in
Medical
Imaging

SURVEY OF 90+ PAPERS

 Medical
imaging models
 Pre-trained networks with
Transfer learning
 U-Net, V-Net, E-Net
 FCN - fully convolutional net
with skip connections, Multi-
stream CNNs
 TieNet, DenseCNN Encoder +
RNN Decoder - Multi-label
classification
 FCN + MDP (RL) for 2d/3d
Image Registration
 Medical
imaging
models
TIENET - AUTOMATIC LABELS FOR
CHEST X-RAYS
Shift towards Specialized Compute

 Special purpose Cloud

 Google TPU, Microsoft Brainwave, Intel Nervana, IBM Power AI, Nvidia v100
 Bare Metal Cloud - Preview AWS, GCE coming April 2018
 Spectrum: CPU, GPU, FPGA, Custom Asics
 Edge Compute: Hardware accelerators, AI SOC
 Intel Neural Compute Stick, Nvidia Jetson, Nvidia Drive PX (Self driving cars)
 Architectures
 Cluster Compute, HPC, Neuromorphic, Quantum compute
 Complexity in Software
 Model tuning/optimizations specific to hardware
 Growing need for compilers to optimize based on deployment hardware
 Workload specific compute: Model training, Inference
CPU Optimizations

 Leverage High Performant compute tools

 Intel Python, Intel Math Kernel Library (MKL),
NNPack (for multi-core CPUs)
 Compile Tensorflow from Source for CPU
Optimizations
 Proper Batch size, using all cores & memory
 Proper Data Format
 NCHW for CPUs vs Tensorflow default NHWC
 Use Queues for Reading Data
Tensorflow CPU Optimizations

 Compile from source

 git clone https://github.com/tensorflow/tensorflow.git
 Run ./configure from Tensorflow source directory
 Select option MKL (CPU) Optimization
 Build pip package for install
 bazel build --config=mkl --copt=-DEIGEN_USE_VML -c opt
//tensorflow/tools/pip_package:build_pip_package
 Install the optimized TensorFlow wheel
 bazel-bin/tensorflow/tools/pip_package/build_pip_package
~/path_to_save_wheel
pip install --upgrade --user ~/path_to_save_wheel /wheel_name.whl
 Intel Optimized Pip Wheel files
Parallelize your models

 Data Parallelism
 Tensorflow Estimator + Experiments
 Parameter Server, Worker cluster
 Intel BigDL Spark Cluster
 Baidu’s Ring AllReduce
 Uber’s Horovod
TensorFusion
 HyperTune Google Cloud ML
 Model Parallelism
 Graph too large to fit on one
machine
 Tensorflow Model Towers
Optimizations for Training

Source: Amazon MxNET

Workload Partitioning

 Different layers have different memory-compute

properties
 Minimize communication time
 Place neighboring layers on same GPU
 Balance workload between GPUs
 Model on left more balanced

 LSTM unrolling: ↓ memory, ↑ compute time

 Encode/Decode:↑ memory
  Define your ML Container locally
Cluster  Evaluate with different parameters in the cloud
 Use EFS / GFS for data storage and sharing across
Optimizations nodes
 Create separate Data processing container
 Mount EFS/GFS drive on all pods for shared
storage
 Avoid GPU Fragmentation problems by bundling
jobs
 Placement optimizations - Kubernetes Bundle
as pods, Mesos placement constraints
 GPU Drivers bundling in container a problem
 Mount as Readonly volume, or use Nvidia-
docker
 Future:
Explainability
 Active research area
 Current Techniques
 Activation Heat Maps

 Saliency Maps
 Reconstruct Image

 t-sne vizualization
 Project Brainwave Source: Microsoft Research Blog

Future: FPGA Hardware Microservices

FPGA Optimizations

Can FPGA Beat GPU Paper:

➢ Optimizing CNNs on Intel FPGA
➢ FPGA vs GPU: 60x faster, 2.3x more energy-
efficient
➢ <1% loss of accuracy

ESE on FPGA Paper:

➢ Optimizing LSTMs on Xilinx FPGA
➢ FPGA vs CPU: 43x faster, 40x more energy-
efficient
➢ FPGA vs GPU: 3x faster, 11.5x more energy-
efficient
Brainwave Compiler Source: Microsoft Research Blog
Future: Neuromorphic Compute

Intel’s Loihi: Brain Inspired AI Chip Neuromorphic memristors

Medical Imaging Open Datasets
 http://www.cancerimagingarchive.net/
 Lung Cancer, Skin Cancer, Breast
Cancer….
 Diabetic
Kaggle Open Retinopathy, Lung
Datasets
Cancer
 https://www.kaggle.com/c/data-science-bowl-
 Kaggle Data Science Bowl 2018

2018
  ISIC Skin Cancer Dataset
https://challenge.kitware.com/#challenge/583f126bcad3a51cc66c8d9a
 Grand Challenges in Medical Image Analysis
 https://grand-challenges.grand-

challenge.org/all_challenges/
 https://github.com/sfikas/medical-imaging-datasets
And more…
Where to start your journey?

 Level 1: Just Starting

 Start with the Kaggle and other Open Competitions
 Use the existing pre-trained networks (like GoogleNet) with the Medical Open Source
data
 Level 2: Intermediate
 Experiment with models specific to Medical Imaging space like U-Net/V-Net
 Combine 3rd party data sets for greater insights
 Level 3: Advanced
 Experiment with building new models from scratch
 Level 4: Mature
 Add feedback loop to your models, learning from outcomes
 Experiment with Deep Reinforcement Learning
 Industrialize the ML/DL Pipeline, shared model repository across company
 Resources
 CBInsights AI in Healthcare Map: https://www.cbinsights.com/research/artificial-intelligence-startups-healthcare/
 DL in Medical Imaging Survey : https://arxiv.org/pdf/1702.05747.pdf
 Unet: https://arxiv.org/pdf/1505.04597.pdf

 Learning to diagnose from scratch exploiting dependencies in labels: https://arxiv.org/pdf/1710.10501.pdf

 TieNet Chest X-Ray Auto-reporting: https://arxiv.org/pdf/1801.04334.pdf
 Dermatologist level classification of Skin Cancer using DL: https://www.nature.com/articles/nature21056
 Tensorflow Intel CPU Optimized: https://software.intel.com/en-us/articles/tensorflow-optimizations-on-modern-intel-
architecture
 Tensorflow Quantization: https://www.tensorflow.org/performance/quantization
 Deep Compression Paper: https://arxiv.org/abs/1510.00149
 Microsoft’s Project Brainwave: https://www.microsoft.com/en-us/research/blog/microsoft-unveils-project-brainwave/
 Can FPGAs Beat GPUs?: http://jaewoong.org/pubs/fpga17-next-generation-dnns.pdf
 ESE on FPGA: https://arxiv.org/abs/1612.00694
 Intel Spark BigDL: https://software.intel.com/en-us/articles/bigdl-distributed-deep-learning-on-apache-spark
 Baidu’s Paddle-Paddle on Kubernetes: http://blog.kubernetes.io/2017/02/run-deep-learning-with-paddlepaddle-on-
kubernetes.html
 Uber’s Horovod Distributed Training framework for Tensorflow: https://github.com/uber/horovod
 TFX: Tensorflow based production scale ML Platform: https://dl.acm.org/citation.cfm?id=3098021
 Explainable AI: https://www.cc.gatech.edu/~alanwags/DLAI2016/(Gunning)%20IJCAI-16%20DLAI%20WS.pdf
 Any
Questions