Bayesian deep learning for seismic facies classification and its uncertainty estimation
Pradip Mukhopadhyay and Subhashis Mallick, Department of Geology and Geophysics, University of Wyoming
Downloaded 06/21/20 to 130.238.7.40. Redistribution subject to SEG license or copyright; see Terms of Use at http://library.seg.org/
SUMMARY
Probabilistic pixel-wise semantic segmentation using
Convolutional neural networks and Monte Carlo sampling
with dropout during testing has gained popularity because of
its ability to obtain uncertainly maps from deep learning
models and reduce model over-fitting when predicting
semantic classes. In this work, we propose an encoder-
decoder based Bayesian SegNet architecture for seismic
facies classification and introduce the concept of predictive
entropy to obtain uncertainty maps. By applying the method
to real seismic data with salt and sediment structures, we
observe high prediction uncertainty at facies boundaries, for
data samples which are affected by processing and imaging
artifacts, and in the zones where the sediments are trapped
within the salt bodies. Comparing Monte Carlo dropout
model prediction with the state of the art SegNet architecture
without dropout, we demonstrate the usability of the
proposed Bayesian SegNet for seismic facies classification
and uncertainty estimation.
INTRODUCTION
With the increased use of artificial intelligence (AI) systems
in real-life scenarios like autonomous driving and medical
diagnosis, convolution neural networks (CNNs), a deep
learning technique is also gaining its popularity in the oil and
gas industry. The CNN models designed for image
segmentation task, known as semantic segmentation require
an understanding of an image at the pixel level. This method
has been used successfully in seismic facies classification in
two-dimensional (2D) seismic profiles or traversing through
all 2D profiles in three-dimensional (3D) data volumes using
both a small patch of data, and with encoder-decoder based
architecture (Zhao, 2018; Badrinarayan et al., 2017; Chen et
al., 2018).
Quantifying uncertainty should be a natural component of
any prediction system, because it is important to know the
confidence with which we can trust our prediction for
decision making. Seismic data poses its own challenges due
to the limitation of available training samples, presence of
incoherent noise, and the noise resulting from the processing
and imaging artifacts. Unlike other computer vision images
where features are well defined within each class, seismic
facies are often nonunique and are contaminated by samples
from other classes because of geologic and tectonic
complexities, and it is important to provide uncertainty maps
along with pixel-wise model predictions which an
experienced seismic interpreter can use for decision making.
© 2019 SEG
SEG International Exposition and 89th Annual Meeting
To measure the prediction uncertainty, Gal and Ghahramani
(2016) introduced dropout as approximate Bayesian
inference over the network’s weight and showed that
dropout can be used during the testing phase to impose a
Bernoulli distribution over the convolution net filter weights
to estimate model uncertainty in conjunction with model
prediction. The dropout technique is also used to regularize
CNNs to prevent model overfitting and co-adaptation of
features. SegNet (Badrinarayan et al., 2015) is a state of the
art and easy to implement CNN based semantic
segmentation architecture which can be trained end-to-end
in one step due to its lower parameterization as compared
with heavily parameterized Fully Convolutional Networks
(FCN) (Long, et al., 2014) and Dilation Network (Yu and
Koltun, 2016). Kendall et al., (2016) used the concept of
Monte Carlo (MC) dropout in the SegNet architecture and
called it Bayesian SegNet. They successfully demonstrated
the practical use of this technique for model prediction and
uncertainty estimation.
The classification prediction uncertainty can be addressed
using epistemic uncertainty, representing what the model
does not know due to insufficient data and aleatoric
uncertainty caused by noisy data measurements. The entropy
of the predictive distribution, also known as predictive
entropy, captures predictive uncertainty which combines
both epistemic and aleatoric uncertainties (Gal, 2016). We
introduced this predictive entropy measurement in Bayesian
SegNet architecture to measure model prediction uncertainty
while classifying seismic facies. Although the deep learning
prediction uncertainty estimation using dropout has been
successfully studied in computer vision applications, it has
not been introduced and extensively studied in geoscience
applications.
In this work, we use the encoder-decoder based Bayesian
SegNet model to classify seismic facies from real seismic
data. We present the classification results along with
prediction uncertainty maps by measuring predictive
entropy and demonstrate the possible sources of high
uncertainty in our predictions. Finally, we conclude that
modelling uncertainty using MC dropout also improves
segmentation performance by minimizing model overfitting.
BAYESIAN CONVOLUTIONAL ENCODER-
DECODER NEURAL NETWORK
The encoder-decoder based CNN consists of a sequence of
non-linear processing layers or encoders and a
10.1190/segam2019-3216870.1
Page 2488
 Seismic facies classification using Deep Neural Network
Downloaded 06/21/20 to 130.238.7.40. Redistribution subject to SEG license or copyright; see Terms of Use at http://library.seg.org/
corresponding set of decoders followed by a pixel-wise
classifier. In general, each encoder unit consists of one or
more convolutional layers with batch normalization and a
rectified linear unit (ReLU), followed by non-overlapping
max-pooling and sub-sampling. Next, in the decoder process,
the spatial dimensions are gradually recovered using
deconvolution and up-sampling where the max-pooling
indices in the encoding sequence are utilized. In computer
vision terminology this process is known as the semantic
segmentation and is widely used for visual scene
identifications.
Deep neural networks with model prediction and uncertainty
quantification capabilities are commonly known as Bayesian
neural network. As performing inference in Bayesian neural
network is difficult, variational inference is used to model
the posterior. Gal and Grahramani (2015) introduced
dropout as approximate Bayesian inference over network’s
weights and linked it to variational inference in deep neural
network with Bernoulli distributions over the network’s
weight. This is achieved by sampling the network with
randomly dropped out unit during testing in a fashion similar
to MC sampling is carried out from the posterior
distributions. In CNN architecture, the posterior distribution
over the convolutional weights w given the observed training
data x and labels y as 𝑝(𝑤|𝐱, 𝑦) is not tractable. Hence, the
distribution of these weights is approximated using
variational inference as 𝑞(𝑤) via minimizing the Kullback-
Leibler (KL) divergence between this approximate
distribution and the full posterior as:
KL(𝑞(𝑤) || 𝑝(𝑤|𝐱, 𝑦) (1)
In equation 1, the approximating variational distribution
𝑞(𝑤𝑖 ) for every K x K dimensional convolutional layer i,
with units j, is defined as:
b𝑖,𝑗 ~ Bernoulli(pi ) for j = 1, … , 𝐾𝑖,
𝑤𝑖 = M𝑖 diag(b𝑖 ) (2)
with b𝑖 vectors of Bernoulli distributed random variables, pi
the dropout probability and variational parameters M𝑖 , we
obtain the approximate model of the Gaussian process.
© 2019 SEG
SEG International Exposition and 89th Annual Meeting
Bayesian SegNet is a popular probabilistic encoder-decoder
network architecture where the model is trained by inserting
dropout in the training process and the distribution over the
weights is sampled during testing using Monte Carlo
stochastic sampling to obtain the posterior distribution of
softmax class probabilities (Kendell, et al., 2016). Figure 1
shows the schematic of the Bayesian SegNet architecture
used in this work. We inserted dropout layers only in the
middle flow of SegNet, as the low-level features in shallow
layers of the network are mostly consistent across the
distribution of the models and hence can be represented
using deterministic weights whereas the higher-level
features in the deep layers are better modelled using
probabilistic weights.
A Bayesian model for semantic segmentation not only
produces model predictions for each pixel but also generates
pixel-wise uncertainty estimates. Among others, the
prediction model uncertainty can be measured using
predictive entropy. The predictive entropy captures
predictive uncertainty which combines both epistemic and
aleatoric uncertainty (Gal, 2016).
The predictive entropy H ̂ [𝑦|𝐱, 𝐷𝑡𝑟𝑎𝑖𝑛 ] given a test input x
and the training data 𝐷𝑡𝑟𝑎𝑖𝑛 can be approximated as:
̂ [𝑦|𝐱, 𝐷𝑡𝑟𝑎𝑖𝑛 ] =
H
1 1
− ∑ ( ∑ 𝑝(𝑦 = 𝑐|𝐱, 𝑤
̂ 𝑡 ) log( ∑ 𝑝(𝑦 = 𝑐|𝐱, 𝑤
̂ 𝑡 )) (3)
T T
c 𝑡 𝑡
where y is the output variable, c ranges over all the classes,
T is the number of Monte Carlo samples (stochastic forward
passes), 𝑝(𝑦 = 𝑐|𝐱, 𝑤̂ 𝑡 ) is the softmax probability of input x
being in class c, and 𝑤̂ 𝑡 are the model parameters on the tth
Monte Carlo sample. The predictive entropy attains its
maximum value when all classes are predicted to have equal
and uniform probability, and its minimum value when one
class has probability 1 and all others a probability of 0 (i.e.
the prediction is certain).
10.1190/segam2019-3216870.1
Page 2489
 Seismic facies classification using Deep Neural Network
APPLICATION OF BAYESIAN CNN
To demonstrate the application of Bayesian CNN for pixel-
wise seismic facies classification and uncertainty
quantification, we used stacked seismic data where each
pixel is labelled as either salt or sediment classes. The data
Downloaded 06/21/20 to 130.238.7.40. Redistribution subject to SEG license or copyright; see Terms of Use at http://library.seg.org/
used in this study were prepared by TGS and available for
academic research.
A seismic image is the representation of the digital signal
amplitude recorded by sensors and are generated by the
source waveform interactions at the elastic impedance
boundaries between different rock formations. These images
show the amplitude difference of different rock boundaries
but does not provide quantitative geological interpretation of
each rock-type. The impedance contrast within the facies
represented by salt structure is not significant whereas the
contrasts between the boundaries of salt and sediment layers
is high and produce strong reflected signal amplitudes.
Finally, the sediment layers within the salt bodies possess
other imaging challenges as incident signal becomes
obstructed due to low illumination angles. The main goal of
this image-based classification is to automatically identify
salt and sediment facies from seismic images and measure
the prediction uncertainty using Machine Learning
technique, and we used Bayesian SegNet to achieve this goal.
We used 4000 training images and their corresponding
classes, labelled as either salt or sediment to train our
Bayesian network. Each training image had 101x101 pixels.
We used batch normalization layers after each convolution
layer and trained the network with median frequency class
balancing using the method proposed by Eigen and Fergus
(2014). The MC dropout was inserted in the middle layers.
For training, we used stochastic gradient descent method
with base learning rate of 0.001 and weight decay parameter
set to 0.0005. We continued the process with dropout until
convergence when no further improvements of training
losses were recorded.
Following training, 1000 test images were fed into the
network to obtain model prediction and uncertainty
estimation. We used 200 stochastic forward passes with
dropout to estimate per-pixel model prediction and
uncertainty. We used the mean of the softmax probability as
a measure of final model prediction. We also recorded mean
model accuracy, per class accuracy, and predictive entropy
from Equation 3. In Figure 2, we present some qualitative
results on the test images. The samples marked as red are
classified as salt and the grey samples are classified as
sediments. We achieved about 92% prediction accuracy
using Bayesian SegNet architecture. This result confirms
that Bayesian CNN technique can be used for seismic facies
classification from stacked seismic images. To benchmark
our Bayesian SegNet algorithm with the state of the art
SegNet architecture, we use the same training samples to
train SegNet algorithm and tested against all 1000 images.
Table 1 shows the seismic facies prediction accuracy using
Bayesian SegNet with and without dropout. This result
© 2019 SEG
SEG International Exposition and 89th Annual Meeting
showed that we achieved performance improvement of
about 4% using Monte Carlo sampling with dropout,
compared with a standard SegNet architecture without
dropout (see Table 1). Thus, the results confirm that
modeling uncertainty using Monte Carlo sampling with
dropout not only produces uncertainty maps but also helps
reducing network over-fitting and achieve superior
predictions.
The knowledge of confidence with which we can rely on our
model prediction is important for any decision-making
system and this is especially vital for geological
interpretations as we use passive measured data to interpret
or distinguish geological facies. Deep neural network is the
widely accepted Machine Learning architecture used for
regression, natural clustering and classification problems.
The practical use of Deep neural network in geoscience
applications poses some naturally occurring errors and
limitations, such as model over-fitting, prediction error due
to insufficient training samples and incoherent noise present
on data that are caused by processing artifacts. In Figure 2.
darker color of predictive entropy indicates higher prediction
uncertainty. We observe high uncertainty inside the
boundaries of the facies. Geological facies boundary is the
region where different sedimentary layers change
gradationally from one into another with a mixture of
mineralogical compositions. Therefore, high uncertainty
inside the facies boundaries makes is geological sense. We
also observe higher uncertainty (marked as green boundaries
in Figure 2) where the samples are either contaminated by
processing or imaging artifacts, and where there are
sediments trapped inside salt body.
CNN Mean
accuracy (%)
With MC dropout 91.56
Without dropout 87.14
Table 1. Seismic facies classification accuracy with and
without dropout.
CONCLUSION
In this study, we used Bayesian CNN models for seismic
facies classification and uncertainty estimation. The
Bayesian model clearly classifies the salt and sediment
facies from stacked seismic amplitude image and produces
prediction uncertainty estimates. Our study also shows that
Bayesian CNN is a useful architecture for facies
classification with limited training samples. We achieved
superior prediction performance using Monte Carlo
sampling with dropout by reducing model over-fitting
compared to a normal CNN architecture. The uncertainty
maps produced by our method are valuable and can provide
guidance to an experienced interpreter to use machine
generated facies classes in production environments. In
10.1190/segam2019-3216870.1
Page 2490
 Seismic facies classification using Deep Neural Network

future, we plan to extend this classification method in a

continuous 2D or 3D seismic sections to bring more
meaningful interpretations of our predictions.

ACKNOWLEDGEMENT
Downloaded 06/21/20 to 130.238.7.40. Redistribution subject to SEG license or copyright; see Terms of Use at http://library.seg.org/

The authors thank TGS for sharing the classified seismic

data for academic research and publication. The CNN
models discussed in this study were implemented in
TensorFlow, an open source library from Google, and using
the SegNet architecture.

© 2019 SEG 10.1190/segam2019-3216870.1

SEG International Exposition and 89th Annual Meeting Page 2491
 REFERENCES
Badrinarayanan, V., A. Kendall, and R. Cipolla, 2015, SegNet: A deep convolutional encoder-decoder architecture for image segmentation: ar-
Xiv:1511.00561, 2, 3, 4, 5, 6, 9.
Chen, L. C., Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, 2018, Encoder-decoder with atrous separable convolution for semantic image seg-
mentation: arXiv preprint, arXiv:1802.02611v2.
Eigen, D., and R. Fergus, 2014, Predicting depth, surface normal and semantic labels with a common multi-scale convolutional architecture: ar-
Downloaded 06/21/20 to 130.238.7.40. Redistribution subject to SEG license or copyright; see Terms of Use at http://library.seg.org/

Xiv:1411.4734, 5, 6.
Gal, Y., 2016, Uncertainty in deep learning: Ph.D. thesis, Cambridge University.
Gal, Y., and Z. Ghahramani, 2016, Dropout as a Bayesian approximation: Representing model uncertainty in deep learning: arXiv:1506.02142v6.
Kendall, A., V. Badrianrayanan, and R. Cipolla, 2016, Bayesian SegNet: Model uncertainty in deep convolutional encoder-decoder architectures for
scene understanding: arXiv:1511.02680v2.
Long, J., E. Shelhamer, and T. Darrell, 2014, Fully convolutional networks for semantic segmentation: arXiv:1411.4038, 1, 2, 5, 6, 8.
Mukhoti, J., and Y. Gal, 2018, Evaluating Bayesian deep learning methods for semantic segmentation: arXiv:1811.12709.
Yu, F., and V. Koltun, 2016, Multi-scale context aggregation by dilated convolutions: ICLR, 2, 6, 8.
Zhao, T., 2018, Seismic facies classification using different deep convolution neural network: 88th Annual International Meeting, SEG, Expanded
Abstracts, 2046–2049, doi: 10.1190/segam2018-2997085.1.
Zhao, T., and P. Mukhopadhyay, 2018, A fault detection workflow using deep learning and image processing: 88th Annual International Meeting,
SEG, Expanded Abstracts, 1966–1969, doi: 10.1190/segam2018-2997005.1.

© 2019 SEG 10.1190/segam2019-3216870.1

SEG International Exposition and 89th Annual Meeting Page 2492