Multi-Classification of Brain Tumor Images

Using Convolutional Neural Network

Himanshu Jain Sahil Upgalanwar Grace George
School of Computer School of Computer School of Computer
Engineering and Technology, MIT Engineering and Technology, MIT Engineering and Technology, MIT
World Peace University - 411038 World Peace University - 411038 World Peace University - 411038
Pune, India Pune, India Pune, India
engghjain76@gmail.com sahilupaganlawar9@gmail.com ggh1999@gmail.com

Omkar Temgire Shridevi Karande

School of Computer
Engineering and Technology, MIT
World Peace University -411038
Pune,India
omkartemgire12@gmail.com
School of Computer
Engineering and Technology, MIT
World Peace University - 411038 Pune , India
shridevi.karande@mitwpu.edu.in

Abstract
Brain tumor classification is a crucial task to evaluate
the tumors and make a treatment decision according to
their classes.A Deep-learning paradigm that has
achieved substantial results in image segmentation and
classification is the convolutional neural network (CNN).
A new CNN architecture for brain tumor classification
was presented in this study. The classification was
performed using a T1-weighted contrast-enhanced MRI
image database which contains three tumor types
namely Meningioma , Glioma And Pituitary and images
with no brain tumor . The model has proved to provide
an overall validation accuracy and training accuracy by
94.07% and 97.78 % without 5-fold cross validation ,
94.98% and 93.676% with 5-fold cross validation. The
paper summarizes the importance of different factors
involved in performing Hyperparameter Tuning with the
experimentational observations and inferences drawn
from the same alongwith the purpose of k fold cross
validation answering to the question that why are we
performing the same .With good generalization
capability and good execution speed, the new developed
CNN architecture could be used as an effective
decision-support tool for radiologists in medical
diagnostics.
Keywords : Brain Tumor, MRI, CNN, Data
Augmentation, K- fold Cross Validation,
multi-classification
I . INTRODUCTION
Cancer is the second biggest cause of mortality
worldwide, according to the World Health
Organization (WHO)[1]. Cancer can be detected
early and prevented, although this is not always
practicable. A tumour, unlike cancer, can be
benign, pre-carcinoma, or malign. Benign
tumours are distinguished from malignant
tumours in that they do not spread to other organs
or tissues and can be surgically removed [2].
Gliomas, meningiomas, and pituitary tumours are
examples of primary brain cancers. Gliomas are
tumours that grow from tissues other than nerve
cells and blood vessels in the brain. Meningiomas,
on the other hand, come from the membranes that
protect and surround the brain and central nervous
system, whereas pituitary tumours are lumps
inside the skull [3–6]. The most significant
distinction between these three tumour forms is
that meningiomas are usually benign, but gliomas
are usually malignant. Unlike meningiomas,
which are slow-growing tumours, pituitary
tumours, even if benign, can induce various
medical problems. Because of the preceding facts,
it's critical to distinguish between these three
forms of tumours. The exact distinction between
these three types of cancers is a critical step in the
clinical diagnosis process and subsequent
effective patient assessment. Magnetic resonance
imaging is the most used approach for tumour
type differential diagnosis (MRI). However, it is
vulnerable to human subjectivity, and human
observation of a vast amount of data is
challenging. Early brain–tumor identification is
mostly dependent on the radiologist's experience
[7]. The tumor's diagnosis could not be complete
without determining whether it is benign or
malignant. A biopsy is frequently performed to
determine whether the tissue is benign or
cancerous. A biopsy of a brain tumour is not
routinely acquired before definitive brain surgery,
unlike cancers elsewhere in the body [8]. It is
critical to build an effective diagnostics tool for
tumour segmentation and classification from MRI
images [7] in order to acquire exact diagnostics
and prevent surgery and subjectivity .
New technologies, particularly artificial
intelligence and machine learning, have had a
considerable impact on the medical sector,
serving as a crucial support tool for a variety of
medical professions, including imaging. In MRI
image processing, various machine-learning
approaches for image segmentation and
classification are used to give radiologists a
second view.
Deep Learning (DL) is a subdivision of ML that is
based on learning data representations and
hierarchical feature learning. DL algorithms
utilize the arrangement of numerous layers of
nonlinear processing identities for feature
extraction. The output of each sequential layer is
the input of the next one, and that helps in data
abstraction as we go deep within the network [9].
Convolutional Neural Network (CNN) is a class
of DL and commonly used in analyzing visual
imagery and designed to require minimal
preprocessing [10]. It is inspired by biological
processes in the human brain [11] and utilized to
handle data that come in multiple arrays [12]. The
main advantages of CNNs are feature learning and
providing unlimited accuracy rather than
traditional machine learning and vanilla neural
networks which may be achieved by increasing
training samples and therefore leads to a more
robust and accurate model [11]. In CNN
architecture, the convolutional filters are acting as
feature extractors, and as we go deep, we extract
more and more complex features (spatial and
structural information). Feature extraction
happens through convolving small filters with the
input patterns followed by selection of the most
distinguishing features and then starting to train
the classification network [12].
In this paper we intend to present a proper
working model in order to do multi class
classification for detecting the type of Brain
tumors (i.e. Meningioma , Glioma , Pituitary and
No tumor ) Using Deep Learning model based on
Convolutional Neural Networks. The model was
developed using two approaches - by creating a
VGG16 model and by using the 5 fold Cross
Validation about which is explained in detail in
Section 2. The model performance was tested
using the VGG 16 model and in that we have
created different versions of the model in order to
build a comparative study which is explained in
detail in Section 3. We have used 5 fold
Cross-Validation to further increase our model’s
performance so that the model can perform better
with unseen data. The details of this have been
explained in Section 2.3.1 and results are
discussed in Section 4.
II. METHODOLOGY
A. Image Database
The Image Database is taken from kaggle [13]
where the dataset is divided into 4 folders namely
Pituitary Tumor , Meningioma Tumor , Glioma
Tumor and No Tumor . The number of images
present in the database for Pituitary Tumor is 926
images similarly for Meningioma Tumor is 375
images , for Glioma Tumor it is 927 images and
the patient not having any kind of tumor is ( i.e
No Tumor ) 433 images . The images present in
the database are T1-weighted contrast MRI
images also they are present in three different
views namely Axial , Coronal and Sagittal which
is represented by Figure 1 . Figure 2 represents
the three classes of tumors namely Meningioma ,
Glioma and Pituitary Tumor .
Figure 1. Pituitary tumor is demonstrated in three different acquisition views (Axial, Coronal, and
Sagittal) from left to right respectively. Tumors are localized inside a red rectangle.

Figure 2. Different three axial brain tumor types as follows; Meningioma, Glioma and Pituitary tumor from
left to right respectively . Tumors are localized inside a red rectangle

B. Image Pre-Processing to the three intensities required for each pixel

In order to perform image preprocessing we
have implemented the following techniques
in our model so that the feature extraction
involved in the Convolutional Neural
Network becomes uniform and easy . Firstly ,
Resizing of Brain MRI images is performed
as the Magnetic Resonance Images obtained
from the database were of different sizes .
These images represent the input layer of the
network so they were normalized using One
Hot Encoding and resized into two respective
image sizes 128 * 128 pixels and 150 * 150
pixels and as per the experimentation results
obtained the validation accuracies are
monitored about which a detailed description
is given in Section 3 . Secondly , Grayscaling
is performed on the Brain MRI images . In
RGB space, a 'gray' colour is one in which
the red, green, and blue components have all
the same intensity.As a result, each pixel only
requires a single intensity value, as opposed
in a full colour image. The importance of
Grayscaling lies in the fact that it reduces the
complexity of the model as , consider using
RGB images of 10x10x3 pixels to train
neural articles [14] .There will be 300 input
nodes in the input layer. For grayscale photos,
on the other hand, the same neural network
will only require 100 input nodes.
C. K Fold Cross Validation
Due to sample variability between training
and test set, our model gives a better
prediction on training data but fails to
generalize on test data. This leads to a low
training error rate but a high test error rate. To
solve the two issues we use an approach called
cross-validation. Cross Validation is a
technique which involves reserving a
particular sample of a dataset on which you do
not train the model.
 All of our data is used in testing our model,
Later, you test your model on this sample thus giving a fair, well-rounded evaluation
before finalizing it. Before attempting cross metric.
validation there are few things that need to be
considered . Firstly, we should train the model K-fold cross-validation may lead to more
on a large portion of the dataset. Otherwise accurate models since we are eventually
we’ll fail to read and recognise the underlying utilizing our data to build our model.
trend in the data. This will eventually result in
a higher bias. Secondly, we also need a good
But there are also some disadvantages linked
ratio of testing data points. As we have seen
with it like - The computing power is high.
above, less amount of data points can lead to a
So it may take some time to get feedback on
variance error while testing the effectiveness
the model’s performance in the case of large
of the model. Thirdly, we should iterate on the
data sets.
training and testing process multiple times.
We should change the train and test dataset
distribution. This helps in validating the model Slower feedback makes it take longer to find
effectiveness properly. the optimal hyperparameters for the model.

In our model we have used [15] 5 fold cross D . Proposed CNN Architecture
validation.The steps to perform cross
validation are -

1. Randomly split your entire dataset into

k”folds”

2. For each k-fold in your dataset, build your

model on k – 1 folds of the dataset. Then, test
the model to check the effectiveness for kth
fold

3. Record the error you see on each of the

predictions

4. Repeat this until each of the k-folds has

served as the test set.

5. The average of your k recorded errors is

called the cross-validation error and will serve
as your performance metric for the model.

There are some major advantages when we

apply k fold cross validation on our model like
as mentioned below .

K-fold cross-validation works well on small

and large data sets.

Figure 3. Module Diagram of the system

The system is divided into 3 modules, the input
after passing through each module gives the
final output of the model. The first module is the
‘Preprocessing Module’ which consists of
processes involved in making the input image
quality for the model more enhanced than the
initial input fed to the system. This module
consists of tasks such as
● Data Augmentation - in this the dataset
size is increased by performing
operations such as inverting image,
rotating image etc.
● Grayscaling and Thresholding - in this
grayscaling and thresholding techniques
are used and applied on the images
which will in turn increase the overall
model efficiency.
● Salt Noise - salt noise modification is
done on the input images so that the
problem of overfitting can be handled
better.
The second module is the ‘CNN Module’ which
consists of processes which together make our
CNN model’s complete execution such as
● Train/Test Split - in this the input
database is split for training and testing
purposes
● Layer Architecture - in this the complete
layer architecture of the system is
residing. The details about the layer
structure is explained in Table no 1.
● K fold Cross Validation - in our model
we have taken the value of K as 5.
The third and the final module in our system is
the performance module, this consists of
processes which will determine how well the
model performs such as loss analysis, accuracy
analysis and Hyperparameter Tuning.
Input Layer - The input layer is responsible
for confirming the scale of input images as
well as applying data normalisation.
Convolutional Layer - 2D convolutional
layer moves K convolutional filters (kernels)
of size (M N) along the input images and
computes the dot product of the weights
(kernels weights) and the input.
RELU Layer - Following each convolutional
layer is a non-saturated activation function
called ReLU, which is mostly used to reduce
training time as compared to other activation
functions.
Cross Channel Normalisation Layer - A
cross-channel normalisation layer scales and
adjusts the related activations to normalise
the input layer. It creates a local response
normalization layer based on a channel-wise
with a specific size window .
Max Pooling Layer - The pooling layer is
used to minimise the number of parameters
and, as a result, network computations.
Dropout Layer - In this layer, several
activations (nodes) are dropped out at
random, which speeds up the training
process considerably.
Fully Connected Layer -This layer connects
every neuron in one layer to every neuron in
the layer above and below it.
Softmax Layer - The Softmax layer is used
to squash all expected classes between 0 and
1, resulting in a total sum of 1 number.
Classification Layer - Finally, we use a
classification layer to estimate the
classification loss and provide the final
expected categorical mark for each input
image. This layer is based on cross - entropy
loss.
 Table1. CNN Layer Network . All Layers Listed with Properties

After the Training of the model is completed Training and Validation Accuracy as 97.78
over the layer architecture as shown in Table and 94.076 respectively where all the layers
1 which represents the final version of model are listed along with the noticeable properties
that we created after performing of the respective layer the model is classified
Hyperparameter Tuning which gave us on the four classes namely No_tumor ,
Meningioma_Tumor , Glioma_Tumor and
Pitiutary_Tumor .
III . Hyperparameter Tuning Observations
We have carried out an experimentation study
with a mix of conventions that the Deep
Learning practitioner’s follow around the
world which gives us a good starting point as
to how the below factors affect the overall
accuracy of the Brain Tumor Multi Class
Classification Model or a general CNN
model where we have kept the version 2 of
the model as our basis of comparative
study[18].
A. Image Size
With taking Image size into consideration we
have tested our model 2 by taking in two
values as (128 , 128) pixels and (150 , 150)
pixels and we got the validation accuracy as
92.728 and 92.6829 respectively . The
conclusion that we can draw from here is that
Image Size as a factor shown by the results
mentioned above does not make much of a
difference while training a CNN model . The
only point we need to take care about is that
all the images may it be part of Training set ,
Validation set or Test set need to be of similar
size and also while creating a Web
application where we are taking input as a
image from the end user also needs to be
resized on the basis of the image size used so
as to load that particular Web application .
B. Batch Size
With taking Batch size into consideration
we have tested our model 2 by taking in the
values as 16 , 40 , 64 and 128 for which we
generalization capacity of the neural network,
the model should be trained for an optimal
number of epochs. A part of training data is
got the validation accuracies as 91.986 ,
94.076 , 92.728 and 89.895 respectively .
Batch size serves to be an important factor
in order to do Hyperparameter Tuning for
which we have to carry out multiple
experimentations in order to find an optimal
number which is suitable for your model
which in our case was 40 [16]. The
conclusion we can draw from here is that in
general Deep Learning practitioners want
there batch size to be on a higher side like
(128 , 256 or 1024 ) so that there model
takes computationally less time to train but
having a higher Batch Size often leads to
asymptotically dropping down of accuracy
and also we do not want to have Batch Size
on the lower side like 16 because that would
take a lot of time in order to train your
model computationally . Deep Learning
practitioners use Batch Size - 64 as a good
starting point which later can be tuned to
find the optimal number by multiple
experimentations and observations .
C. Number Of Epochs
With taking Number of Epochs into
consideration one of the critical issues while
training a neural network on the sample data
is Overfitting. When the number of epochs
used to train a neural network model is more
than necessary, the training model learns
patterns that are specific to sample data to a
great extent. This makes the model incapable
of performing well on a new dataset.This
model gives high accuracy on the training set
(sample data) but fails to achieve good
accuracy on the test set. In other words, the
model loses generalization capacity by
overfitting to the training data.To mitigate
overfitting and to increase the
dedicated for validation of the model, to
check the performance of the model after
each epoch of training. Loss and accuracy on
the training set as well as on validation set
are monitored to look over the epoch number
after which the model starts overfitting. In
our case we monitored the same by using the
Early Stopping callback function provided in
the keras library where you can provide the
parameter which you want to monitor that is
either loss or accuracy. If the loss is being
monitored, training comes to halt when there
is an increment observed in loss values. Or, If
accuracy is being monitored, training comes
to halt when there is decrement observed in
accuracy values.
D. Optimization Function
With taking different Optimization Functions
into consideration we have tested our model 2
by using Adam Optimizer , RMSprop and
Stochastic Gradient Descent[17] for which we
got the validation accuracies as 92.728 ,
89.1986 and 39.024 respectively .The
conclusion we can draw from here is that
Adam Optimizer as an optimization function
performs the best in less time and more
efficiently as shown by the above results .
Most deep learning practitioners also use
Adam optimizer as a good starting point for
their model.
E. Dropout Rate
Figure 5 . (a) Shows Loss Vs Number of Epochs and (b) Shows Validation Accuracy Vs Number of Epochs ,
for the Final CNN model trained after Hyperparameter Tuning.
While taking dropout rate into consideration
the main issue arises when you increase
dropout beyond a certain threshold, it results
in the model not being able to fit properly.
Intuitively, a higher dropout rate would result
in a higher variance to some of the layers,
which also degrades training. Dropout is like
all other forms of regularization in that it
reduces model capacity. If you reduce the
capacity too much, it is sure that you will get
bad results.The solution is to not use high
dropouts. If you must, lowering the learning
rate and using higher momentum may
help[18].
F. Learning Rate
While taking Learning rate into
consideration we have tested our model 2 by
taking in the value as 0.01 , 0.001 and
0.0001 for which in case of value as 0.01 as
we trained the model with using early
stopping and because the loss was not
asymptotically decreasing and the accuracy
was not increasing so the process got
terminated in between . In case of 0.001 we
got validation accuracy as 92.728 and for
0.0001 we got validation accuracy as
87.804[20] . The conclusion we can draw
from here is that the optimal value for
learning rate in order to train our model is
0.001.
 IV. Results and Discussion
Results of the Final version of the VGG 16
model trained after performing Hyperparameter
Tuning are visualized using the graphs
represented in Figure 5 . Where (a) shows the
graph between Loss and the numbers of epochs
wherein as the number of epochs increases the
loss decreases . Figure 5 (b) shows the graph
between Validation Accuracy and the Number
of epochs wherein as the number of epochs
increases the validation accuracy also increases .
Finally the Training and Validation accuracy
achieved by training this model is 97.78 % and
94.07 % respectively . Now as we discussed
that Normal VGG 16 model in certain cases
does not perform well on unseen data as it was
discussed in Section 2.3.1 therefore we applied
5 fold cross validation in order to use our
training data more efficiently and the final
Training and Validation Accuracy we got after
taking mean of the scores of all 5 folds is
93.676 % and 94.982572 % respectively .
V. Tools And Time Consumption
The proposed deep neural network structure is
trained on Intel i5 9th Gen - 9300H CPU (2.4
GHz with Turbo Boost Upto 4.1 GHz ),
NVIDIA GTX 1650 Ti With Max-Q (4 GB)
GPU, 8GB RAM, Jupyter Notebook and
Python. For Creating the Diagrams of the
Architecture we used draw.io . The training time
was 71 minutes approximately incase of training
the model without 5 fold cross validation and
incase of with 5 fold cross validation it took 185
minutes approximately.
VI. Conclusion
A new CNN architecture for brain tumor
classification was presented in this study. The
classification was performed using a
T1-weighted contrast-enhanced MRI image
database which contains three tumor types. As
input, we used whole images, so it was not
necessary to perform any preprocessing or
segmentation of the tumors. Our designed
neural network is simpler than pre-trained
networks, and it is possible to run it on
conventional modern personal computers. In
this project we have automated the diagnosis
procedure for the brain tumor detection by the
use of Deep Learning. Apart from several
existing brain tumor segmentation and
detection methodology present for MRI of
brain image our project has proved to provide
an overall validation accuracy and training
accuracy by 94.07% and 97.78 % without
5-fold cross validation , 94.98% and 93.676%
with 5-fold cross validation. In this paper, we
have proposed a system that can be used for
segmentation of brain MRI Images for
Detection and identification of brain tumors.
We find a type of tumor that is Meningiomas ,
Gliomas or Pituitary Tumor . The paper
summarizes the importance of different factors
involved in performing Hyperparameter
Tuning with the experimentational
observations and inferences drawn from the
same alongwith the purpose of k fold cross
validation answering to the question that why
are we performing the same .
 VII. References
[1] World Health Organization—Cancer.
Available online:
https://www.who.int/news-room/fact-sheet
s/detail/ cancer. (accessed on 5 November
2019).
[2] Priya, V.V. , “An Efficient
Segmentation Approach for Brain Tumor
Detection in MRI.”, Indian J. Sci. Technol.
2016, vol 9, pp. 1–6.
[3] Cancer Treatments Centers of
America—Brain Cancer Types. Available
online: https://www.cancercenter.
com/cancer-types/brain-cancer/types
(accessed on 30 November 2019).
[4] American Association of Neurological
Surgeons—Classification of Brain
Tumours. Available online:
https://www.aans.org/en/Media/Classificati
ons-of-Brain-Tumors (accessed on 30
November 2019).
[5] DeAngelis, L.M. , “Brain Tumors”.
New Engl. J. Med. 2001, 344, 114–123.
[6] Louis, D.N. , Perry, A. , Reifenberger,
G. , Von Deimling, A. , Figarella-Branger,
M. , Cavenee, W.K. , Ohgaki, H. ,
Wiestler, O.D. , Kleihues, P. , “ Ellison,
D.W. The 2016 World Health Organization
Classification of Tumors of the Central
Nervous System”, A summary. Acta
Neuropathol. 2016, 131, 803–820.
[7] Afshar, P. , Plataniotis, K.N. ,
Mohammadi, “A. Capsule Networks for
Brain Tumor Classification Based on MRI
Images and Coarse Tumor Boundaries.”,
In Proceedings of the ICASSP 2019–2019
IEEE International Conference on
Acoustics, Speech and Signal Processing
(ICASSP), Brighton, UK, 12–17 May
2019; pp. 1368–1372.
[8] Byrne J. , Dwivedi R.; Minks D.
“Tumours of the brain.” In Nicholson T
(ed) Recommendations Cross Sectional
Imaging Cancer Management, 2nd ed.;
Royal College of Radiologists: London,
UK, 2014; pp. 1–20. Available online:
https://www.rcr.ac.uk/publication/recomm
endations-cross-sectional-imaging-cancer-
managementsecond-edition (accessed on 5
November 2019).
[9] L. Deng and D. Yu, ‘‘Deep learning:
Methods and applications ’’, Found.
Trends Signal Process., vol. 7, nos. 3–4,
pp. 197–387, Jun. 2014.
[10] Y. LeCun. (2015). Lenet-5,
Convolutional Neural Networks.
Accessed: May 2019. [Online]. Available:
http://yann.lecun.com/exdb/lenet
[11] M. Matsugu, K. Mori, Y. Mitari, and
Y. Kaneda, ‘‘Subject independent
facial expression recognition with robust
face detection using a convolutional neural
network,’’ Neural Netw., vol. 16, nos. 5–6,
pp. 555–559, Jul. 2003.
[12] Y. LeCun, Y. Bengio, and G. Hinton,
‘‘Deep learning,’’ Nature, vol. 521,
no. 7553, p. 436, 2015 .
[13] Kaggle Dataset . Available online:
https://www.kaggle.com/sartajbhuvaji/brai
n-tumor-classification-mri
[14] Carl F. Sabottke, Bradley M. Spieler.
“The effect of image resolution on Deep
Learning in Radiography”. Available
Online:
https://doi.org/10.1148/ryai.2019190015
[15] Daniel Berrar , “Cross Validation”.
Available online:
https://www.researchgate.net/publication/3
24701535_Cross-Validation
[16] Ibrahem Kandel, Mauro Castelli.
“The effect of batch size on the
generalizability of the convolutional neural
networks on a histopathology dataset.”
Available online:
https://www.sciencedirect.com/science/arti
cle/pii/S2405959519303455
[17] Techniques to improve accuracy using
data redesigning, Hyper Parameter Tuning
and Model Optimization. Available online:
https://towardsdatascience.com/the-quest-o
f-higher-accuracy-for-cnn-models-42df5d7
31faf
[18] Importance of Dropout Rate changes
in model with respect to change in dropout
Available online:
https://stats.stackexchange.com/questions/
291779/why-accuracy-gradually-increase-t
hen-suddenly-drop-with-dropout
[19] Understanding learning rate and how
it improves performance in deep learning.
Available online:
https://towardsdatascience.com/understand
ing-learning-rates-and-how-it-improves-pe
rformance-in-deep-learning-d0d4059c1c10