Professional Documents
Culture Documents
Chapter 6
Histopathological Image
Analysis in Medical
Decision Making:
Classification of Histopathological
Images Based on Deep Learning Model
R. Meena Prakash
Sethu Institute of Technology, India
ABSTRACT
Digital pathology is one of the significant methods in the medicine field to diagnose
and treat cancer. The cell morphology and architecture distribution of biopsies
are analyzed to diagnose the spread and severity of the disease. Manual analyses
are time-consuming and subjected to intra- and inter-observer variability. Digital
pathology and computer-aided analysis aids in enormous applications including
nuclei detection, segmentation, and classification. The major challenges in nuclei
segmentation are high variability in images due to differences in preparation of slides,
heterogeneous structure, overlapping clusters, artifacts, and noise. The structure of
the proposed chapter is as follows. First, an introduction about digital pathology and
significance of digital pathology techniques in cancer diagnosis based on literature
survey is given. Then, the method of classification of histopathological images using
deep learning for different datasets is proposed with experimental results.
DOI: 10.4018/978-1-5225-6316-7.ch006
Copyright © 2019, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
Histopathological Image Analysis in Medical Decision Making
INTRODUCTION
140
Histopathological Image Analysis in Medical Decision Making
141
Histopathological Image Analysis in Medical Decision Making
BACKGROUND
142
Histopathological Image Analysis in Medical Decision Making
images which is essential for analyzing the tumour. Stromal tissue includes the fatty
and fibrous connective tissues surrounding the blood vessels, ducts and lobules, and
lymphatic vessels. The spatial arrangement of stromal cell in tumors is a prognostic
factor in breast cancer. The method employed two alternating convolutional layers,
max pooling layers, two full connection layers, and a final classification layer. Xu
et al. (2013) proposed multi-label classification for Colon Cancer in which four
kinds of features – Color Histogram, Gray-Level Co-occurrence Matrix (GLCM),
Histogram of Oriented Gradients (HOG), and Euler Number are introduced in the
feature set. The four different classes are well and moderately differentiated tubular
adenocarcinoma (HM), moderately and poorly differentiated tubular adenocarcinoma
(LM), Signet-ring carcinoma (LR) and mucinous adenocarcinoma (RMu). Yonekura
et al (2017) proposed a deep convolutional network for classification of glioma
histopathological images of brain tumours and demonstrated an accuracy of 87.2%.
Zheng et al (2017) proposed nucleus-guided feature extraction framework based
on convolutional neural network for classification of histopathological images. The
nuclei are first detected from images and then used to train the CNN. The dataset
used in the method comprised of 2 class dataset with malignant and benign breast
tumours and 15-class dataset with 13 sub categories of breast tumours.
METHODS
l −1 L −1
(i − µ )( j − µ )
Correlation = ∑∑G (i; j )
i j
(1)
i =0 j =0
σi σ j
143
Histopathological Image Analysis in Medical Decision Making
G represents the gray level value of the image of each pixel. µ and σ represent
the mean and standard deviation.
L −1 L −1
L −1 L −1
Contrast = ∑∑ (i − j ) G (i; j )
2
(3)
i =0 j =0
4. Variance: Variance is the deviation of the intensity values of the pixels from
the mean and is a measure of the contrast in the image.
L −1 L −1
Variance = ∑∑ (i − µ) G (i; j )
2
(4)
i =0 j =0
144
Histopathological Image Analysis in Medical Decision Making
The basic processing element of the neural network is the neuron which receives an
input, processes it, and generates an output which is sent to other neurons and further
processed or it is the final output. The inputs to the neuron are multiplied by weights
and these weights are updated during the training process. Another linear component
bias is added to the result of input-weight product. Then a nonlinear function called
activation function is applied to the linear combination of inputs to obtain the final
output. The commonly used activation functions are sigmoid functions, Rectified
Linear Units, Softmax etc., Neural networks are the backbone of deep learning and
they are formed by numerous interconnected neurons. The input layer is the first layer
of the neural network which receives the input. The hidden layers perform specific
tasks on the input data and pass the output to the next layer. The output layer is the
final layer of the network. There will be multiple neurons present in each layer and
all the neurons in each layer are connected to all the neurons in the next layer. It is
called Multi Layer Perceptron (MLP) and the networks are called Fully Connected
Layers (Belsare, 2012; Chatterjee et al, 2017; Das et al, 2014).
The accuracy of the network is measured using cost or loss function, for example,
mean squared error and the learning process strives to minimize the cost. The
optimization algorithm used to minimize the cost is Gradient Descent method. The
rate at which the network descends towards the minima of the cost function is called
the learning rate. The selection of learning rate should be optimal since a too large
learning rate will result in missing of the optimal solution and if the learning rate
is too small, it will be difficult for the network to converge. In a neural network,
initially random weights and bias values are assigned to the nodes. As the output is
received after first iteration, the network error is calculated and is fed back to the
network along with the gradient of the cost unction. The weights are then updated
such as to minimize the error in the subsequent iterations and this is called back
propagation. During the training of neural network, the input is divided into several
batches of equal size randomly. A single training iteration of all the batches in both
forward and back propagation is called an epoch. Increasing the number of epoch will
increase the accuracy, but setting too high will take longer time to converge and the
network might be over-fit. During the training process, a certain number of neurons
is randomly dropped to prevent over-fitting of the network which is called dropout.
Also, the data is normalized at each layer before passing it to the next layer which
is called batch normalization to ensure that distribution of data remains unchanged.
145
Histopathological Image Analysis in Medical Decision Making
f l is used to perform convolution with filter banks, pooling and non-linear activation.
With the gradient descent method, the optimized weight vector is calculated by
M
1
Opt (v1, v2 , …..vl ) = ∑Ω( f ( x ; v , v , ….v ), y )
i
l
i
(6)
M i =1
1 2
' Ω ' denotes the loss function, ‘M’ is the number of data.
146
Histopathological Image Analysis in Medical Decision Making
EXPERIMENTAL SET UP
Dataset
Two datasets are used in the experiment. The KIMIA Path960 dataset comprises
of images from 20 scans of muscle, epithelial and connective tissue with different
texture and pattern types. From each scan, 48 regions of interest are selected and
downsampled to 308x168 patches. Hence, the whole dataset consists of 960 images.
Figure 2 shows the sample images from first 9 scans of the dataset. It is assessed
that though large texture variability exists between the classes, some inter-class
similarities exist which may affect the classification. Also, intra class variability
exists in the dataset images.
The UCSB Bio-Segmentation Benchmark dataset consists of 58 breast cancer
histopathological images categorized as 36 beningn tumour cells and 28 malignant
tumour cells (Gelasca et al, 2008). Figure 3 shows the sample images of UCBS dataset.
Top row - beningn tumor images and bottom row - malignant tumour images
The Deep Convolutional Neural Network employed in the proposed work comprises
of three convolutional layers, three batch normalization layers, three rectified linear
unit layers, two max pooling layers, one fully connected layer, softmax layer and a
final classification layer. The convolutional and the pooling layers produce feature
147
Histopathological Image Analysis in Medical Decision Making
EXPERIMENTAL RESULTS
The performance of CNN is evaluated by training the CNN with 80% of the input data
and 20% of the input data is used for testing. The input color images of size 308x168
are converted to gray scale images and down sampled to half the size (154x84) and
fed as input to CNN. The downsampled gray scale images are shown in Figure 4.
Table 1 shows the classification accuracy obtained on KimiaPath960 dataset.
Table 1 illustrates the results obtained for the classification of images of “Kimia
Path960” dataset. Figure 5 illustrates the training and classification stages of CNN.
The results are compared with that of the segmentation accuracy obtained with
Support Vector Machine classifier subsequent to GLCM feature extraction. The
GLCM features – Correlation, Contrast, Entropy and variance are extracted from
the input gray scale images and used for training and classification using SVM.
Figure 6 illustrates the classification using SVM.
148
Histopathological Image Analysis in Medical Decision Making
149
Histopathological Image Analysis in Medical Decision Making
150
Histopathological Image Analysis in Medical Decision Making
Table 3. Comparison of results for Breast cancer histo dataset - CNN and SVM
CONCLUSION
REFERENCES
Al-Kofahi, Y., Lassoued, W., Lee, W., & Roysam, B. (2010). Improved Automatic
Detection and Segmentation of Cell Nuclei in Histopathology Images. IEEE
Transactions on Biomedical Engineering, 57(4), 841–852. doi:10.1109/
TBME.2009.2035102 PMID:19884070
Belsare, A. (2012). Histopathological Image Analysis Using Image Processing
Techniques: An Overview. Signal and Image Processing: an International Journal,
3(4), 23–36. doi:10.5121ipij.2012.3403
Chatterjee, S., Dey, N., Shi, F., Ashour, A. S., Fong, S. J., & Sen, S. (2017). Clinical
application of modified bag-of-features coupled with hybrid neural-based classifier
in dengue fever classification using gene expression data. Medical & Biological
Engineering & Computing, 56(4), 709–720. doi:10.100711517-017-1722-y
PMID:28891000
151
Histopathological Image Analysis in Medical Decision Making
Das, H., Jena, A. K., Nayak, J., Naik, B., & Behera, H. S. (2014). A Novel PSO
Based Back Propagation Learning-MLP (PSO-BP-MLP) for Classification. In
Computational Intelligence in Data Mining (Vol. 2, pp. 461–471). Smart Innovation,
Systems and Technologies.
Dey, N., & Ashour, A. S. (2018). Computing in Medical Image Analysis. Soft
Computing Based Medical Image Analysis, 3-11.
Gelasca, E. D., Byun, J., Obara, B., & Manjunath, B. (2008). Evaluation and benchmark
for biological image segmentation. 2008 15th IEEE International Conference on
Image Processing.
Han, Z., Wei, B., Zheng, Y., Yin, Y., Li, K., & Li, S. (2017). Breast Cancer Multi-
classification from Histopathological Images with Structured Deep Learning Model.
Scientific Reports, 7(1). PMID:28646155
Haralick, R. M., Shanmugam, K., & Dinstein, I. (1973). Textural Features for Image
Classification. IEEE Transactions on Systems, Man, and Cybernetics, SMC-3(6),
610–621. doi:10.1109/TSMC.1973.4309314
Irshad, H., Veillard, A., Roux, L., & Racoceanu, D. (2014). Methods for Nuclei
Detection, Segmentation, and Classification in Digital Histopathology: A Review—
Current Status and Future Potential. IEEE Reviews in Biomedical Engineering, 7,
97–114. doi:10.1109/RBME.2013.2295804 PMID:24802905
Jia, Z., Huang, X., Chang, E. I., & Xu, Y. (2017). Constrained Deep Weak Supervision
for Histopathology Image Segmentation. IEEE Transactions on Medical Imaging,
36(11), 2376–2388. doi:10.1109/TMI.2017.2724070 PMID:28692971
Komura, D., & Ishikawa, S. (2018). Machine Learning Methods for Histopathological
Image Analysis. Computational and Structural Biotechnology Journal, 16, 34–42.
doi:10.1016/j.csbj.2018.01.001
Kumar, M. D., Babaie, M., Zhu, S., Kalra, S., & Tizhoosh, H. R. (2017). A
comparative study of CNN, BoVW and LBP for classification of histopathological
images. 2017 IEEE Symposium Series on Computational Intelligence (SSCI).
10.1109/SSCI.2017.8285162
Pan, X., Li, L., Yang, H., Liu, Z., Yang, J., Zhao, L., & Fan, Y. (2017). Accurate
segmentation of nuclei in pathological images via sparse reconstruction and
deep convolutional networks. Neurocomputing, 229, 88–99. doi:10.1016/j.
neucom.2016.08.103
152
Histopathological Image Analysis in Medical Decision Making
Saha, M., Chakraborty, C., & Racoceanu, D. (2018). Efficient deep learning model
for mitosis detection using breast histopathology images. Computerized Medical
Imaging and Graphics, 64, 29–40. doi:10.1016/j.compmedimag.2017.12.001
PMID:29409716
Sirinukunwattana, K., Raza, S. E., Tsang, Y., Snead, D. R., Cree, I. A., & Rajpoot,
N. M. (2016). Locality Sensitive Deep Learning for Detection and Classification of
Nuclei in Routine Colon Cancer Histology Images. IEEE Transactions on Medical
Imaging, 35(5), 1196–1206. doi:10.1109/TMI.2016.2525803 PMID:26863654
Vieira, S., Pinaya, W. H., & Mechelli, A. (2017). Using deep learning to investigate
the neuroimaging correlates of psychiatric and neurological disorders: Methods and
applications. Neuroscience and Biobehavioral Reviews, 74, 58–75. doi:10.1016/j.
neubiorev.2017.01.002 PMID:28087243
Xu, J., Luo, X., Wang, G., Gilmore, H., & Madabhushi, A. (2016). A Deep
Convolutional Neural Network for segmenting and classifying epithelial and stromal
regions in histopathological images. Neurocomputing, 191, 214–223. doi:10.1016/j.
neucom.2016.01.034 PMID:28154470
Xu, Y., Jiao, L., Wang, S., Wei, J., Fan, Y., Lai, M., & Chang, E. I. (2013). Multi-label
classification for colon cancer using histopathological images. Microscopy Research
and Technique, 76(12), 1266–1277. doi:10.1002/jemt.22294 PMID:24123468
Yonekura, A., Kawanaka, H., Prasath, V. B., Aronow, B. J., & Takase, H. (2017).
Improving the generalization of disease stage classification with deep CNN for Glioma
histopathological images. 2017 IEEE International Conference on Bioinformatics
and Biomedicine (BIBM). 10.1109/BIBM.2017.8217831
Zheng, Y., Jiang, Z., Xie, F., Zhang, H., Ma, Y., Shi, H., & Zhao, Y. (2017). Feature
extraction from histopathological images based on nucleus-guided convolutional
neural network for breast lesion classification. Pattern Recognition, 71, 14–25.
doi:10.1016/j.patcog.2017.05.010
153