CSCI 1850: Deep Learning in Genomics

Spring 2021
http://cs.brown.edu/courses/csci1850

Mar 25, 2021

Thursday

Instructor: Ritambhara Singh

Format: Online (Synchronous)
Time: TTh 10:30-11:50 AM
Today’s outline
Section IV: Interpretation of deep learning models

• DeepMotif Dashboard: Visualization methods for TFBS predictions

o Saliency Map (muddy points)
o Temporal outputs
o Class based optimization

• Class based optimization in DeepChrome

• Class Activity

• AttentiveChrom: Prediction gene expression from HMs using attention

 Muddy point: S+ ?
Saliency map

Image courtesy: https://pdfs.semanticscholar.org/b82d/2edf99303589ec40e87a91e4d08470baf743.pdf

 Muddy point: Calculation
Saliency map

Image courtesy: https://pdfs.semanticscholar.org/b82d/2edf99303589ec40e87a91e4d08470baf743.pdf

 Muddy point: Can we use saliency maps to choose
Saliency map where to perform perturbation analysis?

Image courtesy: https://www.geeksforgeeks.org/what-is-saliency-map/

Questions?
Temporal output

What neural network

architecture could this
be related to?

Image courtesy: https://pdfs.semanticscholar.org/b82d/2edf99303589ec40e87a91e4d08470baf743.pdf

Temporal output

Image courtesy: https://pdfs.semanticscholar.org/b82d/2edf99303589ec40e87a91e4d08470baf743.pdf

Temporal output

Image courtesy: https://pdfs.semanticscholar.org/b82d/2edf99303589ec40e87a91e4d08470baf743.pdf

Temporal output

Image courtesy: https://pdfs.semanticscholar.org/b82d/2edf99303589ec40e87a91e4d08470baf743.pdf

DeepMotif Dashboard Setup

Image courtesy: https://pdfs.semanticscholar.org/b82d/2edf99303589ec40e87a91e4d08470baf743.pdf

DeepMotif Dashboard Results

“..the fact that there is no clear JASPAR motif in this sequence

may show that the traditional motif approach is not always the
best way to model TFBSs…
Image courtesy: https://pdfs.semanticscholar.org/b82d/2edf99303589ec40e87a91e4d08470baf743.pdf
Questions?
Visualization methods until now

How does the sequence affect the score?

What sequence will result in a certain class?

Image courtesy: https://pdfs.semanticscholar.org/b82d/2edf99303589ec40e87a91e4d08470baf743.pdf

Class based optimization

Image courtesy: https://pdfs.semanticscholar.org/b82d/2edf99303589ec40e87a91e4d08470baf743.pdf

Class based optimization

Image courtesy: https://pdfs.semanticscholar.org/b82d/2edf99303589ec40e87a91e4d08470baf743.pdf

Class based optimization

Image courtesy: https://pdfs.semanticscholar.org/b82d/2edf99303589ec40e87a91e4d08470baf743.pdf

Class based optimization

Image courtesy: https://pdfs.semanticscholar.org/b82d/2edf99303589ec40e87a91e4d08470baf743.pdf

Questions?
DeepChrome: Model

HM1
HM2 5. Softmax
HM3
f(X)
HM4
HM5 Y=0/1
X
1. Convolution 2. Max 3. Dropout 4. Multi-layer
Pooling Perceptron
DeepChrome: Class based optimixation
Trained model

HM1
HM2 5. Softmax
HM3
f(X)
HM4
HM5 Y=1
X
1. Convolution 2. Max 3. Dropout 4. Multi-layer
Pooling Perceptron
DeepChrome: Class based optimixation
Trained model

HM1
HM2 5. Softmax
HM3
f(X)
HM4
HM5 Y=1
X
1. Convolution 2. Max 3. Dropout 4. Multi-layer
Pooling Perceptron
DeepChrome: Class based optimization

Color Scale
Y= 0 0.0 1.0

Cell-type: E057
H3K27me3
H3K36me3
H3K4me1
H3K4me3
H3K9me3
Freq. of active bins
PROMOTER
DISTAL PROMOTER
REPRESSOR
Questions?
Class activity [10 mins]

• Think
Compare and contrast the following visualization methods
Method Advantages Disadvantages

CNN filter weights

Perturbation analysis

Saliency maps

Temporal output

• Pair Class based optimization

• Share:
https://docs.google.com/document/d/12IRDRjYk272UNoQB8QUMqIURu9Dk-
iPB3UyZqpFYL8Q/edit?usp=sharing
Questions?
AttentiveChrome
Interpretation using attention

Attention
Mechanism

“Park”

HM1 (1) What positions are important?

DNA Gene
(2) What HMs are important? Gene
HM2
Expression
DNA 28
Gene
Formulation of attention in neural networks (NLP)

Courtesy: https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html
Formulation of attention in neural networks (NLP)

Courtesy: https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html
Formulation of attention in neural networks (NLP)

Courtesy: https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html
Formulation of attention in neural networks (NLP)

Courtesy: https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html
Formulation of attention in neural networks (NLP)

Courtesy: https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html
Formulation of attention in neural networks (NLP)

Courtesy: https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html
Formulation of attention in neural networks (NLP)

Courtesy: https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html
AttentiveChrome: Setup

HM-Level (2) What HMs are important?

Attention
Gene
Expression

Bin-Level (1) What positions are important?

Attention

Input
HM1 HM2 HM3
AttentiveChrome: Recurrent Neural Network

HM1 HM1
AttentiveChrome: Attention

Attention weights per gene

Softmax

W
Context
Vector

HM1 HM1
AttentiveChrome: Performance

0.95

Area Under ROC Curve (AUC) Score

0.9

0.85

0.8

0.75

0.7

0.65

0.6

0.55
Improvement for 49/56 0.5
56 Cell-types
Cell-types
RFC SVC DeepChrome AttentiveChrome
AttentiveChrome: Bin-level attention
(1) What positions are important?
CELL TYPE: GM12878 (Blood Cell)
Repressors Promoters

H3K27me3 H3K9me3 H3K4me1 H3K4me3 H3K36me3

AttentiveChrome: HM-level attention
(2) What HMs are important?
Cell Types: H1-Hesc GM12878 K562
(Stem Cell) (Blood Cell) (Leukemia) Color Scale

H3K27me3

H3K36me3

H3K4me1

H3K4me3
PROMOTER
DISTAL PROMOTER
REPRESSOR H3K9me3

Gene: PAX5
Questions?
Upcoming
Course website: http://cs.brown.edu/courses/csci1850

Section IV : Interpretation of deep learning models

• March 30: DeepLIFT: feature attribution method

• Homework 3 released, due April 01 (Thursday) at 11:59PM

Wrap up
https://forms.gle/GzgVkGkiSNzxj92k6

What was the clearest point today?

What was the muddiest point today?