Expert Systems With Applications 238 (2024) 122159

Contents lists available at ScienceDirect

Expert Systems With Applications

journal homepage: www.elsevier.com/locate/eswa

Development of hybrid models based on deep learning and optimized

machine learning algorithms for brain tumor Multi-Classification
Muhammed Celik *, Ozkan Inik
Tokat Gaziosmanpasa University, Engineering and Architecture Faculty, Computer Engineering Department, Tokat 60250, Turkey

A R T I C L E I N F O A B S T R A C T

Keywords: Accurate classification of magnetic resonance imaging (MRI) images of brain tumors is crucial for early diagnosis
Brain tumors and effective treatment in clinical studies. In these studies, many models supported by artificial intelligence (AI)
Deep learning have been proposed as assistant systems for experts. In particular, state-of-the-art deep learning (DL) models that
Machine learning
have proven themselves in different fields have been effectively used in the classification of brain MRI images.
Hyperparameter optimization
Classification
However, the low accuracy of multiple classification of these images still leads researchers to conduct different
studies in this field. Especially there is a need to develop models that achieve high accuracy on original images,
and it is believed that this need can be met not only by DL models but also by classical machine learning (ML)
algorithms. However, it is critical to choose the hyperparameters correctly for the hybrid use of ML algorithms
with DL models. This study proposes a powerful new hybrid method to perform multiple classifications of brain
tumors with high accuracy. This method also uses a novel convolutional neural network (CNN) model for feature
extraction, and ML algorithms are used for feature classification. In addition, nine state-of-the-art CNN models
are used for CNN performance comparison. The Bayesian optimization algorithm is used to obtain the optimal
hyperparameter values of ML algorithms. The results obtained from the experimental studies show that the
proposed hybrid model achieved 97.15% mean classification accuracy and 97% recall, precision, and F1-score
values. Other hybrid models, including DarkNet19-SVM, DarkNet53-SVM, DenseNet201-SVM, EfficientNetB0-
SVM, InceptionV3-SVM, NasNetMobile-SVM, ResNet50-SVM, ResNet101-SVM, and Xception-SVM, achieved
mean classification accuracies of 95.01%, 95.58%, 96.87%, 97.01%, 95.3%, 95.01%, 96.3%, 95.87%, and
96.23%, respectively. Additionally, the proposed hybrid model exhibited remarkable time efficiency, accom­
plishing the classification process in a mere 67 min. Conversely, the model that exhibited the lowest time effi­
ciency was the InceptionV3, with a processing time of 370 min. In terms of computational complexity, the
EfficientNetB0 model is the most efficient. Despite the higher computational complexity of the proposed CNN
model compared to some other models, it achieves the second-best classification accuracy. These results show
that the proposed method performs better than previous studies on the same dataset. Especially in the classifi­
cation problem, the optimized ML algorithms were superior to CNN classifiers. Finally, except for one, the
proposed CNN model achieved better classification accuracies than the state-of-the-art CNN models.

1. Introduction
The brain is a complex and essential organ that controls numerous
vital functions, composed of billions of neurons, synapses, and nerve
cells. However, like other organs, the brain is also prone to developing
abnormalities known as brain tumors. These tumors refer to the
abnormal growth of cells in different sizes and locations within the brain
and can be fatal by killing healthy cells (Abd-Ellah et al., 2018). Brain
tumors are categorized into two types: benign and malignant. Benign
tumors are usually not fatal as they do not spread to other tissues. In
contrast, malignant tumors are fast-growing and can spread to other
tissues, severely affecting the quality of life (Kumar and Mankame,
2020). Brain tumors comprise several common types, including glioma,
meningioma, and pituitary, which can have fatal outcomes (Badža and
Barjaktarović, 2020). Gliomas refer to a group of tumors that develop in
the glial cells, the supportive tissue of the brain. They account for
approximately 30 % of all brain tumors and are often malignant. Me­
ningiomas are primarily benign and slow-growing tumors that arise

* Corresponding author.
E-mail addresses: muhammed.celik@gop.edu.tr (M. Celik), ozkan.inik@gop.edu.tr (O. Inik).

https://doi.org/10.1016/j.eswa.2023.122159
Received 15 February 2023; Received in revised form 10 September 2023; Accepted 12 October 2023
Available online 18 October 2023
0957-4174/© 2023 Elsevier Ltd. All rights reserved.
M. Celik and O. Inik Expert Systems With Applications 238 (2024) 122159

from the brain’s membranes. Although they do not originate from inside Table 1
the brain, they cause symptoms by exerting pressure on the brain as they Recent studies for the classification of brain MRI images.
grow. Pituitary tumors are unusual growths that occur in the pituitary Authors Problem Method Dataset Accuracy
gland located at the base of the brain behind the nose. Some of these (%)
tumors can cause the pituitary gland to produce excessive hormones that (Cheng et al., Multi- SVM Figshare (Cheng, 91.28
regulate essential bodily functions and are typically benign (Sharif et al., 2015) Classification 2017)
2020). (Zacharaki Multi- SVM and Special 85
The most common methods used to detect cancerous cells and tissues et al., 2009) Classification KNN
(El-Dahshan Binary- ANN and Harward Medical 98
in the body are medical imaging methods, with MRI and computed to­ et al., 2010) Classification KNN School Dataset (
mography scans being the most frequently used (Khan et al., 2020). MRI Johnson and
is an imaging method that uses magnetic fields and radio waves to Becker, 1999)
obtain high-resolution images, and is especially useful in visualizing the (Zhang et al., Binary- ANN Harward Medical 100
2011) Classification School Dataset
brain, spinal cord, joints, and internal organs, as well as diagnosing and
(Sultan et al., Multi- CNN Figshare 96.13
monitoring diseases such as cancer. MRI provides more precise and 2019) Classification
detailed results compared to other imaging methods, which makes it a (Aamir et al., Multi- CNN Figshare 98.95
useful tool in detecting and diagnosing brain tumors (Díaz-Pernas et al., 2022) Classification
2021; Kumar and Mankame, 2020). Experts evaluate the MRI images to (Chattopadhyay Binary- CNN and BraTS 2020 ( 99.74
and Maitra, Classification SVM Menze et al., 2015)
determine whether the patient has a brain tumor and, if so, which type 2022)
of tumor it is. However, manual examination of MRI images can be time- (Nayak et al., Multi- CNN Figshare 98.78
consuming, costly, and prone to errors, especially with a large volume of 2022) Classification
data (Gómez-Guzmán et al., 2023). This can result in a longer diagnostic (Wahlang et al., Binary- CNN, 1. Figshare 88
2022) Classification CNN- 2. Brainweb (
process, and increased risk to the patient’s life. To overcome these is­
DNN and Collins et al.,
sues, computer-aided diagnosis systems have been developed, and SVM 1998)
studies (Díaz-Pernas et al., 2021; Gómez-Guzmán et al., 2023; Khan 3. Radiopaedia (
et al., 2020; Sharif et al., 2020) have shown that ML and DL are suc­ radiopaedia.org)
cessful methods for classifying, segmenting, and detecting brain tumors. (Al-Badarneh Binary- ANN and Harward Medical 100
et al., 2012) Classification KNN School Dataset
CNN models and DL approaches have been widely used to analyze
(Raza et al., Multi- CNN Figshare 99.6
medical images of malignant tumors and various diseases (Das et al., 2022) Classification
2019), such as pneumoconiosis(Devnath et al., 2022), skin cancer (Alam (Maqsood et al., Multi- CNN- 1. BraTS 2018 ( 98.92
et al., 2022), and brain tumors (Srinivas et al., 2022). However, there are 2022) Classification SVM Menze et al., 2015)
2. Figshare
hybrid frameworks where ML and DL methods are used together (El-
(Amran et al., Binary- CNN Br35H (Hamada, 99.51
Dahshan et al., 2010; Zhang et al., 2011). In order to achieve this, the 2022) Classification 2020)
transfer learning (TL) method is utilized (Deepak and Ameer, 2019). By (Samee et al., Multi- CNN Figshare 99.51
utilizing the weights of a previously trained network, TL can speed up 2022) Classification
the process by providing a shorter training time for the new problem and (Latif et al., Multi- CNN- BraTS 2018 96.19
2022) Classfication SVM
increasing accuracy (Aurna et al., 2022). Studies on brain MRI images in
(Gómez- Multi- CNN 1. Figshare 81.05
the literature are presented in the next section. Guzmán et al., Classification 2. SARTAJ (
2023) Bhuvaji et al.,
1.1. Related works 2020)
3. Br35H
(combination of 3
In this section, studies on the classification of brain tumors in MRI dataset)
images are presented. A thorough search was conducted for recent ar­ (Yazdan et al., Multi- Multi- SARTAJ 91.2
ticles that achieved high classification accuracy in the Science Direct, 2022) Classification Scale
IEEE Xplore, and Google Scholar databases. The search was performed CNN
(Díaz-Pernas Multi- Multi- Figshare 97.3
using the keywords “Brain MRI Images,” “Deep Learning,” and “Machine et al., 2021) Classification Scale
Learning”. Important studies in the literature found with these keywords CNN
are summarised and presented in Table 1. The table includes the prob­ (Jibon et al., Binary- CNN Harward Medical 96
lem that the studies are based on, the methods used to solve the problem, 2022) Classification School Dataset
(Das et al., Multi- CNN Figshare 94.39
and the dataset used. On the far right of the table are the highest ac­
2019) Classification
curacy values achieved in the test phases of the studies. DL models and (Deepak and Multi- CNN- Figshare 98
ML algorithms have been widely used for the classification of brain tu­ Ameer, 2019) Classification SVM,
mors. Cheng et al. (2015) proposed to improve brain tumor classifica­ CNN-
tion performance by enlarging the tumor region through image KNN
(Deepak and Multi- CNN- Figshare 95.82
expansion and then subdividing it into subregions. They used three Ameer, 2021) Classification SVM
methods such as intensity histogram, Gray Level Co-occurrence Matrix (Aurna et al., Multi- CNN- FigsharevSARTAJ 98.96
(GLCM), and Bag of Words (BOW) for feature extraction and used 2022) Classification Ensemble
annular segmentation in addition to tumor region expansion. They (Ullah et al., Binary- CNN Abhranta 99.33
2022b) Classification Panigrahi (
achieved the best accuracy of 91.28 % using a SVM classifier. Zacharaki
Panigrahi, 2021)
et al. (2009) introduced a system for glioma grade classification that (Mohsen et al., Multi- DNN Harward Medical 96.97
utilized SVM and k-Nearest Neighbor (KNN) methods. The system per­ 2018) Classification School Dataset
formed both multiclassification of different glioma grades and binary (Srinivas et al., Binary- CNN Kaggle (Hemanth 96
classification to differentiate between high and low grades. The study 2022) Classification et al., 2019)
(Ananda Kumar Multi- CNN BraTS (Bhupendra 99.57
yielded an accuracy of 85 % for multiclassification and 88 % for binary et al., 2022) Classification et al., 2022)
classification. El-Dahshan et al. (2010) proposed a method for classi­
fying brain MRI images as normal and abnormal using artificial neural
network (ANN) and KNN classifiers. They used Discrete Wavelet

2
M. Celik and O. Inik
Transform (DWT) and Principal Component Analysis (PCA) methods in
image preprocessing stages. In the study, the highest accuracy of 98 %
was achieved with the KNN classifier. Zhang et al. (2011) proposed an
ANN for binary classification of brain tumor MRI images. They used
DWT and PCA for feature extraction and feature reduction, respectively.
As a result, they achieved 100 % accuracy. Al-Badarneh et al. (2012),
used ANN and KNN methods for automatic binary classification of MRI
images. According to the test results, ANN and KNN achieved 98.92 %
and 100 % accuracy, respectively.
The success of CNN in classification problems is high in medical data
as well as in other data. Sultan et al. (2019) proposed a CNN architecture
for the classification of multi-class brain tumors from MRI images. With
the proposed architecture, they were able to classify images with 96.13
% accuracy. Aamir et al. (2022) proposed three DL models for multiple
classification of brain tumors. Two of them were used for feature
extraction, and one for classification. As a result, the data was classified
with 98.95 % accuracy. Chattopadhyay and Maitra (2022) proposed a
CNN model for classifying MRI images into tumor and non-tumor for
detecting brain tumors. In the proposed method, SVM, softmax, and
sigmoid functions are used in the last layer of the model to diversify the
activation function. RMSProp algorithm was used for optimization. In
the tests, the method using the softmax classifier and RMSProp opti­
mizer gave the best result with 99.74 % accuracy.
In the classification problem of brain tumors, using CNN-based
hybrid structures also provides effective results. These structures
perform the classification process on a new model created by combining
two or more CNN models. Nayak et al. (2022) developed a CNN model
for the classification of brain tumors. This model was derived from the
EfficientNet architecture and was used to classify multiple brain tumor
MRI images. According to the test results, the model achieved 98.78 %
accuracy. In a study conducted by Yazdan et al. (2022) a Multi-Scale
(MS) CNN model was proposed for performing multi-classification on
a four-class brain MRI dataset. According to the results of the study, the
proposed model performed better than ALexNet and ResNet models,
achieving 91.2 % accuracy. Díaz-Pernas et al. (2021) used the MS-CNN
model for the segmentation and multi-classification of brain tumors. The
model was designed by taking inspiration from the processing pathways
in the human visual system and includes analysis of input images at
three different spatial scales. In the study, the proposed model achieved
a tumor classification accuracy of 97.3 %. Wahlang et al. (2022) pro­
posed a CNN model based on the LeNet architecture for detecting brain
tumors in MRI images. Factors such as age and gender were also added
to the model’s criteria in this study. The proposed method was compared
with CNN-DNN, SVM, and AlexNet methods and stood out with the
highest accuracy rate of 88 %. Raza et al. (2022), proposed a model
called DeepTumorNet using the GoogLeNet architecture to perform
multi-classification of three different types of brain tumors. In the pro­
posed method, 15 additional layers were added to the GoogLeNet ar­
chitecture. This model achieved high classification success with a 99.6 %
accuracy rate. Maqsood et al. (2022) developed a method for multi-
classification of brain tumors. This method involves a five-stage
approach. In the first stage, the edges of the input images were detec­
ted. In the second stage, brain tumors were segmented using a 17-layer
CNN model. In the third stage, feature extraction was performed using
the MobileNetV2 CNN model. In the fourth stage, the best features were
selected, and in the final stage, classification was performed using SVM.
The classification process was performed on two separate datasets. As a
result of the experiments conducted, an accuracy of 97.47 % was ob­
tained for the BraTS 2018 dataset and 98.92 % for the Figshare dataset.
Amran et al. (2022), conducted a study using the GoogLeNet architec­
ture for the classification of brain tumors. However, unlike previous
studies, the last 5 layers were deleted, and 14 layers that perform
automatic feature extraction were added in their place. The developed
model was compared with various pre-trained CNN models and machine
learning algorithms. As a result, the proposed model classified brain
tumors with 99.51 % accuracy.
3
Expert Systems With Applications 238 (2024) 122159
In addition to these studies, there are also studies in the literature
that use TL. Latif et al. (2022), proposed a hybrid method for classifying
glioma tumors. The method used a CNN model for feature extraction,
and then classification was performed with SVM.
The success rate obtained from the classification process was calcu­
lated as 96.16 %. Deepak and Ameer (2019) proposed a hybrid method
for classifying brain tumors. The method used an MRI dataset containing
three different brain tumors. GoogLeNet was used as the feature
extractor in the proposed method. These features were then classified
using SVM and KNN algorithms. The results obtained achieved an ac­
curacy of 98 % with a CNN-KNN structure. Deepak and Ameer (2021)
used the CNN-SVM hybrid structure for brain tumor classification, as in
their previous study. The classification result obtained was 95.82 %
accuracy. Jibon et al. (2022) proposed a method for binary classification
of brain MRI images. This method uses log-polar transform (LPT) and
CNN for feature extraction and classification. The proposed method
achieved 96 % accuracy as a result of the study.Mohsen et al. (2018)
proposed combining DWT and D-CNN methods for classifying brain
tumors. DWT was used for feature extraction in this study. Later, the D-
CNN algorithm was used to classify the obtained features. The classifi­
cation result obtained was 96.97 % accuracy. Srinivas et al., (2022)
demonstrated the effectiveness of the transfer learning approach for
binary classification of brain tumors. The study used pre-trained VGG16,
ResNet50, and InceptionV3 models. The highest success rate in the
experimental study was achieved with the VGG16 model at 96 % ac­
curacy. Ananda Kumar et al. (2022) conducted a study using the
ResNet152 pre-trained model to detect and classify benign, malignant,
and healthy brain tissues. Additionally, the CoV-19 OA optimization
algorithm was used for weight parameter optimization. A DCNN model
was used in the data preprocessing stage of the dataset. The proposed
approach achieved 99.57 % accuracy in the test studies.
Studies in the literature demonstrate the effectiveness of using pre-
trained models for brain tumor classification. Gómez-Guzmán et al.
(2023) developed a CNN model for multi-classification of brain tumors.
The performance of various pre-trained models was also compared with
the proposed model. The InceptionV3 CNN model achieved the highest
accuracy on the applied dataset, reaching 97.12 %. The accuracy of the
proposed Generic CNN model was 81.05 %. Das et al. (2019) used his­
togram equalization method in the pre-processing stage of the images in
the dataset used for classification and classified the resulting features
with a CNN model they developed. They achieved an accuracy of 94.39
% in classification. Aurna et al. (2022) used five different pre-trained
CNN models and their proposed CNN model to demonstrate classifica­
tion success. The pre-trained models used were VGG19, EfficientNetB0,
InceptionV3, ResNet50, and Xception. As a result of experiments on the
merged dataset, the proposed model exhibited superior performance
compared to other models, with an accuracy of 98.96 %. Ullah et al.
(2022b) proposed a new CNN model called TumorResnet for binary
classification of brain tumors. The proposed model achieved a classifi­
cation accuracy of 99.33 %.
1.2. Motivation
Early diagnosis of brain tumors is a critical process, and in recent
years, computer vision systems have been used for this diagnosis. Many
studies have been conducted on the classification of brain MRI images,
and DL-based architectures have generally been used in these studies.
However, in some studies, feature extraction has been performed by
CNNs, and classical ML algorithms such as SVM and KNN have been used
for classification. Nevertheless, it has been observed that there are some
shortcomings in the multiple classification of brain MRI images in the
literature. These are listed below in bullet points.
1. The studies on the same dataset show that CNN models are more
successful when used with ML algorithms. However, a single study
should determine which method (CNN or ML) is more successful.
M. Celik and O. Inik
2. In previous studies, the hyperparameters of ML algorithms are usu­
ally used as default, so the impact of parameter optimization on al­
gorithm performance should be investigated.
3. CNNs are typically used with KNN and SVM algorithms, but the
classification performance of other machine learning algorithms
(such as Decision Tree (DT) and Naive Bayes (NB)) should be
explored.
4. State-of-the-art models require high training times when used with
ML algorithms. Therefore, it is necessary to design a new CNN model
to have a shorter training time, so that more efficient results can be
obtained when used with ML.
The main motivation of this study is to investigate each of the above
points one by one and to verify the results with experimental studies.
1.3. Contributions
This study proposes a new approach for classifying four types of
brain tumors (glioma, meningioma, no tumor, and pituitary) using
optimized ML algorithms with a new CNN and state-of-the-art CNN
models. A publicly available dataset (Nickparvar, 2021) was used for the
training and testing processes of the models. This dataset is a combi­
nation of three different datasets, namely Fighshare, SARTAJ, and
Br35H, and contains 7023 images. The contributions of the proposed
method are presented below:
1. Features obtained from nine state-of-the-art CNN models (Dar­
kNet19, DarkNet53, DenseNet201, EfficientNetB0, InceptionV3,
NasNetMobile, ResNet50, ResNet101, and Xception) were fed into
ML algorithms optimized using Bayesian optimization for the clas­
sification task. In addition to feature extraction, these models were
also used for direct classification.
2. To minimize the time complexity in the proposed method, a novel
CNN model was designed in addition to the state-of-the-art models.
The implementation of the developed CNN model with the KNN al­
gorithm resulted in the shortest training time among all the evalu­
ated approaches.
3. The effectiveness of CNN-ML models in MRI classification is
compared in several aspects. One of these aspects is the Model Size
versus Model Accuracy graph. This graph shows the effect of model
size on accuracy as well as other performance measures.
4. The main goal of the study is to discover the optimal hybrid classifier
for the quadruple brain tumor class, which involves a higher
complexity, and this classifier is intended to be used for solving other
multi-class problems.
5. Unlike previous studies on the same or similar datasets, high classi­
fication accuracy was achieved without applying data augmentation
techniques.
The subsequent sections of this paper are structured as follows:
Section 2 elucidates the material and methods employed in the study.
Section 3 presents a detailed account of the proposed method. Section 4
reports on the Experimental results. Discussion of the findings is pre­
sented in Section 5, and finally, a Conclusion is drawn in Section 6.
Fig. 1. Brief of the methodology.
4
Expert Systems With Applications 238 (2024) 122159
2. Material and methods
In this section, the dataset used for brain tumor classification and the
ML algorithms (SVM, KNN, NB and DT) and CNN models used in the
proposed method are discussed. The method, including the proposed
CNN model, is thoroughly explained and visually illustrated in Fig. 1.
The steps summarized in the block diagram will be explained in more
detail in the next section.
2.1. Dataset
In this study, a publicly available MRI dataset is used to train, vali­
date and test CNN-ML based techniques and models. The brain tumor
MRI dataset (Nickparvar, 2021) is a combination of Figshare, SARTAJ
and BrH35 datasets and consists of 4 different classes. The dataset,
detailed in Table 2, is in grayscale and JPG format containing 7023
human brain MRI images of different types. There are four classes of
brain tumors in this dataset: Glioma (1297 images for training and 324
images for testing), Meningioma (1316 images for training and 329
images for testing), No tumor (1600 images for training and 400 images
for testing) and Pituitary (1406 images for training and 351 images for
testing). For training and testing, 80 % and 20 % of the images were
used, respectively. However, resizing was applied to the dataset in the
preprocessing stage to provide a sufficient input size for each model.
Furthermore, the grayscale images were converted to RGB format before
the training and testing phases.
Images belonging to the no-tumor class were taken from the Br35H
dataset. Since the images belonging to the Glioma class in the SARTAJ
dataset were misclassified, these images were taken into account and
replaced when merging the 3 datasets. Fig. 2 shows some sample images
from the dataset.
2.2. State-of-the-art CNN models
In this study, nine state-of-the-art CNN models were used. These
models are widely used and have high accuracy rates in various image
classification tasks. Model selection was based on low parameters and
high accuracy values. Thus, the goal was to achieve high accuracies with
a small model size in MRI image classification. The input image size,
depth, and number of parameters for each model are presented in
Table 3. This information will be helpful in understanding the different
characteristics of these models and evaluating their suitability for a
specific classification task.
Darknet19 model (Redmon and Farhadi, 2016) is a DL model with 19
layers. The model is specifically designed for use in systems with limited
resources such as mobile devices and provides successful results for
object detection, classification and other image processing tasks. The
Table 2
Number of training and test samples for the classes in the dataset.
Class Train Test Total
Meningioma 1316 329 1645
Pituitary 1406 351 1757
Glioma 1297 324 1621
No tumor 1600 400 2000
Total 5619 1404 7023
M. Celik and O. Inik Expert Systems With Applications 238 (2024) 122159

Fig. 2. Some samples of the dataset.

technique called skip connection. This technique allows the network to

Table 3
learn more easily by directly adding the outputs of some layers to deeper
Information about model parameters (Shaukat et al., 2023).
layers. Additionally, batch normalization is used to train the network
CNN Model Input Size Depth Number of Parameters (Million) faster and more effectively. ResNet50 has a network structure with 50
DarkNet19 256x256 19 20.8 convolutional layers and can recognize 1000 different objects trained on
DarkNet53 256x256 53 41.6 the ImageNet dataset.
DenseNet201 224x224 201 18.9 ResNet101 (He et al., 2015) is a deeper and more complex CNN
EfficientNetB0 224x224 82 5.3
InceptionV3 299x299 48 23.9
model derived from ResNet50. This model uses a technique called skip
NasNetMobile 224x224 389 4.4 connections as a solution to the overfitting problems experienced in
ResNet50 224x224 50 25.6 previous ResNet models. ResNet101 has a network structure with 101
ResNet101 224x224 101 44.6 convolutional layers, consisting of convolutional, pooling, and normal­
Xception 299x299 71 22.9
ization layers. Additionally, the softmax function is used at the end of
the network to predict the class of the input image.
input of the model is a 224x224 image and the output is a softmax layer Xception (Chollet, 2016), is a deeper and more complex CNN model
used to identify and classify objects from the ImageNet (Deng et al., using convolutional layers and the depthwise separable convolution
2010) dataset containing 1000 class labels. technique. This technique reduces the computational cost and allows for
Darknet53 (Redmon and Farhadi, 2018) is a 53-layer CNN model better results by performing the convolution in two stages. Xception has
that utilizes a residual network structure, created using convolutional an architecture with 126 convolutional layers. It also uses batch
layers and skip connection blocks that combine the outputs of previous normalization to ensure faster and better training of the network. The
layers. This approach helps to reduce overfitting problems and allows output layers of the network estimate the class of the input image with
for faster training, despite the deeper network structure. Darknet53 is the softmax function.
commonly used as the base model for You Only Look Once V3 (YOLO
V3) in many object detection and classification tasks, with variations 2.3. Feature extraction
trained on different datasets for similar tasks.
DenseNet201 (Huang et al., 2016) is a 201-layer CNN model that Feature extraction is a process used to transform the information in a
heavily utilizes skip connection blocks to create deeper networks while dataset into a more understandable format. In other words, it is a
minimizing overfitting problems. The model includes structures called fundamental pre-processing step for ML algorithms, which usually re­
dense blocks, in which each block learns rich features using all the quires human intervention. However, with DL architectures, automatic
outputs of the previous block as input. DenseNet201 is trained on the feature extraction is performed on raw data, and this is one of the biggest
ImageNet dataset and can recognize 1000 different objects. advantages of DL architectures. While manual extraction requires
EfficientNetB0 (Tan and Le, 2019) is a CNN architecture designed specialized knowledge, automatic extraction does not require human
using the compound scaling technique, which increases the depth, intervention to extract features from signals or images. In particular,
width, and resolution of the network simultaneously to provide high CNNs are used to learn features in images. Each convolution layer
accuracy and efficiency. The EfficientNetB0 model consists of 8 recur­ consists of specialized filters that detect different features in the image.
rent blocks, each containing multiple convolutional layers optimized to The first convolution layers usually learn edges, corners, and other
learn more features using fewer parameters. simple features, while later layers learn more complex features. This
Inceptionv3 (Szegedy et al., 2015) is a CNN architecture that learns process happens automatically, without the need for human interven­
better features with fewer parameters using a special architecture called tion, and helps make images more understandable.
the Inception module. This module allows the network to learn different In this study, nine state-of-the-art CNN models and a new CNN model
features using filters of different sizes. Additional classification layers were utilized to extract features from images. The CNN models consist of
called auxiliary classifiers also enable faster network training. The two main components: feature extraction and classification. Custom
dropout technique helps prevent overfitting. Inceptionv3 can recognize layers of the state-of-the-art CNN models were utilized, as shown in
1000 different objects and is a high-performance model. There are also Table 4, generating 1000 features per image, which helped in reducing
lighter versions of the model that are optimized for use on mobile computational cost. The names of the feature layers in the table belong
devices. to the models in the deep learning library used.
NasNetMobile (Zoph et al., 2017) is a CNN architecture built using
the automated technique known as Neural Architecture Search, which 2.4. Machine learning algorithms
provides high accuracy. The model consists of multiple convolutional
layers and output layers optimized to learn more features using fewer AI is a field within computer science that focuses on developing
parameters. The network is made lighter and faster using a technique techniques, theories, and applications to enable machines to perform
called separable convolution. intelligent tasks. ML, a sub-branch of AI, involves the development of
ResNet50 (He et al., 2015) is a CNN architecture that includes con­ algorithms that can build models based on training data, enabling them
volutional layers in blocks to avoid overfitting problems using a to make predictions or decisions about new data without explicit pro­
gramming (Shaukat et al., 2020). ML has numerous applications,

5
M. Celik and O. Inik Expert Systems With Applications 238 (2024) 122159

Table 4 if-then rules. To construct the tree, entropy and information gain mea­
Information about feature layers of models. sures are used to select the best possible intermediate node (Shaukat
CNN Model Layer type Feature Layer Number of Extracted et al., 2020).
Name Features NB is a type of classifier based on Bayes’ theorem, which breaks
DarkNet19 Average Pooling avg1 1000 down the conditional probability of a problem being analyzed. This
Layer classifier works best with discrete attributes and is considered
DarkNet53 Convolutional conv53 1000 straightforward with a faster detection speed. There are three main
Layer techniques under Naïve Bayes: multinomial, Bernoulli, and Gaussian.
DenseNet201 Fully Connected fc1000 1000
Layer
Multinomial Naïve Bayes is used for discrete values, where feature
EfficientNetB0 Fully Connected head dense 1000 vectors represent the number of occurrences of an event. Bernoulli Naive
Layer MatMul Bayes is used for binary feature vectors, and an example is the “bags of
Inceptionv3 Fully Connected predictions 1000 words” technique. Gaussian Naive Bayes is a classifier used for contin­
Layer
uous values, where the values follow a Gaussian distribution (Shaukat
NasNetMobile Fully Connected predictions 1000
Layer et al., 2020).
ResNet50 Fully Connected fc1000 1000
Layer 3. Proposed method
ResNet101 Fully Connected fc1000 1000
Layer
Xception Fully Connected predictions 1000
In this section, a five-step workflow for a proposed CNN-ML method
Layer for multiclass classification of brain tumors is described, as shown in
Proposed CNN Fully Connected fc_1 1000 Fig. 3. The first step involves providing a dataset consisting of three
Layer different public datasets to the system. In the second step, the dataset is
resized to fit the input dimensions of CNN models. The third step in­
volves training and testing using the proposed CNN model and state-of-
including fault detection (Alharbi et al., 2023) and detection of diseases
the-art CNN models to demonstrate their classification performance. In
(Ali et al., 2022; Kumar et al., 2022).
the fourth step, features extracted from the CNN models are classified by
In the field of machine learning, algorithms can serve three main
ML algorithms with hyperparameter optimization using the Bayesian
purposes: clustering, prediction, and classification. The choice of algo­
optimization algorithm. Finally, the trained models are tested and their
rithm depends on the specific task it is intended to perform. For instance,
performance is evaluated using standard classification metrics such as
SVMs are typically used for classification, while linear regression is
Accuracy, Precision, Recall, F1-score, and AUC. Each step of the pro­
employed for prediction. This study employed four machine learning
posed method is explained in detail in the following sections.
algorithms, namely SVM, KNN, NB, and DT, which will be elaborated on
in this section.
3.1. Proposed CNN model
KNN is a type of learning algorithm that is unsupervised and is used
for clustering problems. It measures how different two pieces of data are
The purpose of this study is to classify brain MRI images into four
by using a distance function. It takes less time to train than other clas­
classes using a newly developed CNN model. The model is capable of
sifiers, but it takes a long time to compute during classification. This
identifying Glioma-0, Meningioma-1, No tumor-2, or Pituitary-3 from an
classifier assumes that similar data points are closer to each other than
input image. It consists of 25 layers, including the input layer, convo­
dissimilar data points. There are two main categories of KNN based on
lution layers with preprocessed (resized) images and ReLU activation
anomaly scores. The anomaly scores can be calculated by finding the
functions, batch normalization, maximum pooling, two fully connected
difference between the kth neighbor and data point or by calculating the
layers, and finally the softmax activation layer. The model architecture
density of each data instance. The value of the kth data point affects the
is illustrated in Fig. 4, and the parameter values for each layer are pre­
overall performance of the classifier. This classifier is sensitive to noisy
sented in Table 5.
data and the distance function used to calculate the difference between
The input layer of the model takes an image of size 224x224 pixels
data points. KNN requires a lot of storage space and is computationally
and has 3 channels including RGB color channels. The first convolution
expensive. The most common distance function used is Euclidean dis­
layer of the model uses 32 filters of size 1x1 and has a stride value of 1
tance (d(x, y)) to find the distance between data points × and y (Shaukat
and a padding value of 0. The second convolution layer also has 64
et al., 2020).
filters of size 1x1. This layer is followed by a pooling layer with a size of
SVM is a popular and successful ML technique used for classification
3x3 and a stride value of 2. The third convolution layer has 64 filters of
problems. It separates and classifies two different data classes based on
size 2x2 with a stride value of 1 and a padding value of 0. The fourth
the margin on either side of the hyperplane. The classifier can achieve
convolution layer has 64 filters of size 3x3 and again has a stride value of
high accuracy in classifying a data point by maximizing the margin and
1 and a padding value of 0. The fifth convolution layer has 128 filters of
distances between hyperplanes. The data points that lie on the border of
3x3 size and has a stride value of 1 and a padding value of 0. This layer is
the hyperplane are called support vector points. SVM is categorized into
followed by another pooling layer, this time with a size of 5x5 and a
two types: linear and non-linear, which depend on the kernel function
stride of 2. The sixth convolution layer has 128 filters of size 3x3 and has
used. Additionally, SVM can be one-class or multi-class, depending on
a stride value of 1 and a padding value of 0. The seventh convolution
the detection type. However, SVM requires a lot of memory for pro­
layer has 128 filters of size 4x4 and again has a stride value of 1 and a
cessing and training time. To learn a dynamic user’s behavior, SVM
padding value of 0. This is followed by another pooling layer with a size
needs training at different time intervals for better results. The perfor­
of 3x3 and a stride of 5. Finally, there are two fully connected (FC)
mance of the classifier is also affected by the kernel function and its
layers, the first with a size of 1x1x1000 and the second with a size of
parameters (Shaukat et al., 2020).
1x1x4.
DT is a supervised ML technique that uses a tree-like structure. The
tree is made up of three parts: a root or intermediate node, path, and leaf
node. The root/intermediate node represents an object/attribute, and 3.2. ML optimization with Bayesian optimization algorithm
each path that diverges from it represents the possible values of the
parent node (object). The leaf node represents the predicted category/ The machine learning algorithms use the extracted features from
classified attribute. The resulting tree is then represented in the form of each CNN model’s feature layers to make classifications. To achieve the
highest possible accuracy in classification, it’s important to identify the

6
M. Celik and O. Inik
Fig. 3. Flowchart of proposed method.
Fig. 4. Proposed CNN model.
factors that affect the accuracy of the algorithms and choose the best
values based on the selected features. In the literature, there are studies
on parameter optimisation of DL models and ML algorithms (Inik, 2023;
Inik and Ulker, 2022; Ini˙k et al., 2021). In this study, hyperparameters of
each ML algorithm were selected through Bayesian optimization and a
standard eight-fold cross-validation procedure during the training pro­
cess. Different ML algorithms have unique hyperparameters that can be
optimized. For example, KNN has hyperparameters like distance metric,
distance weight, and number of neighbors; SVM has hyperparameters
like box constraint level, multiclass method, and kernel type; DT has
hyperparameters like maximum number of splits and split criterion; and
NB has hyperparameters like kernel type and distribution name.
The selection of the split criterion and the maximum number of splits
play a crucial role in the accuracy of the DT classifier. The maximum
number of splits controls the depth of the tree and influences the accu­
racy and generalization of the model. While deeper trees can have high
accuracy rates, they can also be prone to overfitting and may not
perform well on new test sets. On the other hand, shallower trees are
more robust and tend to have similar training accuracy rates as repre­
sentative test sets. The split criterion is used to determine the optimal
splitting point of a node in the tree and can be based on measures such as
the Gini diversity index, Twoing rule, or maximum deviance reduction.
The objective of DT is to produce pure nodes with a single class, and
impurities in nodes can be quantified using metrics such as the Gini
index or deviation criteria. The Twoing rule is an alternative criterion
that maximizes node purity by maximizing the Twoing rule expression.
The type of kernel used in a NB algorithm affects how accurate the
predictions are. There are four options: Gaussian, Box, Epanechnikov,
and Triangle. Each kernel has its own strengths and weaknesses.
Gaussian Naive Bayes models use a Gaussian distribution to calculate the
probabilities of each class, while Kernel Naive Bayes models use a
chosen kernel distribution. The choice of kernel type should depend on
the data and the desired properties of the model. It is important to
7
Expert Systems With Applications 238 (2024) 122159
consider the balance between accuracy and computational complexity
when deciding which kernel to use.
SVM algorithm has three hyperparameters: kernel type, box
constraint level, and multiclass method. The kernel type decides how the
data is transformed before classification and can be linear, Gaussian
Radial Basis Function (RBF), quadratic, or cubic. The box constraint
level controls the range of Lagrange multiplier values, which impacts the
number of support vectors and training time. Selecting these hyper­
parameters carefully is essential for creating an efficient classifier. The
multiclass method converts the multiclass classification problem into
several binary classification subproblems using one-vs-one or one-vs-all
approaches. The one-vs-one method trains a separate SVM learner for
each class pair, while the one-vs-all method trains a single SVM learner
for each class to differentiate it from all other classes.
KNN algorithm has three important settings to be optimized for
achieving the best results: the number of neighbors, distance metric, and
distance weight. The number of neighbors represents how many nearby
neighbors should be considered for classifying each location. The dis­
tance metric calculates the distance between data points, and there are
different options available like Euclidean, City block, and more. The
distance weight changes the importance of each distance in the calcu­
lation, and it can be set to Equal, Inverse, or Squared inverse weighting.
To determine the optimal number of neighbors, a logarithmic search is
conducted in the range of [1,max(2,round(n/2))], where n is the number
of observations. The algorithm searches for the best results in the dis­
tance weight setting as well(Aslan et al., 2022).
Tables 6 and 7 show the hyperparameter values that were found
using Bayesian optimization. These hyperparameter values were then
used to train and test the machine learning algorithm. The training
phase of each algorithm was set to terminate after 20 iterations.
Bayesian optimization is a commonly used and effective method for
hyperparameter optimization. Equation (1) (Wu et al., 2019) represents
the fundamental formula utilized in the Bayesian optimization
M. Celik and O. Inik Expert Systems With Applications 238 (2024) 122159

Table 5 integers, or categorical variables with discrete labels. In hyperparameter

Parameter values at each layer of the proposed CNN model. optimization, f(x) is usually an objective function or a Gaussian Process
Layer Name Activation Learnable Parameters Total Learnable (GP) model, evaluated on a validation set, and X is the set of possible
Maps Parameters values for the hyperparameters. Bayesian optimization is based on
Input 224x224x3 – 0 Bayes’ theorem, which is shown in Eq. (2). Bayes’ theorem uses previous
Conv2D-1 224x224x32 Weights:1x1x3x32, 128 information to make an inference. Eq. (2) shows that the posterior dis­
Bias:1x1x32 tribution is proportional to the likelihood (P(Z|X )) and the prior distri­
ReLu-1 224x224x32 – 0 bution (P(X)), and the belief or forecast is represented by P(X|Z ) (Aslan
Conv2D-2 224x224x64 Weights:1x1x32x64, 2112
Bias:1x1x64
et al., 2022).
ReLu-2 224x224x64 0
–
P(X|Z )∝P(Z|X )P(X) (2)
MaxPool2D-1 111x111x64 – 0
BatchNorm-1 111x111x64 Offset: 1x1x64, 128
Scale: 1x1x64 The process of finding the best hyperparameter in a search space can be
Conv2d-3 110x110x64 Weights:2x2x64x64, 16,448 time-consuming when using the objective function in machine learning
Bias:1x1x64 algorithms. To make the optimization process more efficient, a proxy
ReLu-3 110x110x64 – 0 model is often used to estimate the objective function. Instead of eval­
Conv2D-4 108x108x64 Weights:3x3x64x64, 36,928
Bias:1x1x64
uating the objective function at all points in the search space, iterative
ReLu-4 108x108x64 – 0 methods are used to focus on areas that are more likely to contain the
Conv2d-5 106x106x128 Weights: 3x3x64x128, 73,856 optimal solution, which reduces the number of function evaluations
Bias:1x1x128 needed to find the best hyperparameter (Aslan et al., 2022). Bayesian
ReLu-5 106x106x128 0
–
optimization is a technique that incorporates prior models into the
MaxPool2D-2 51x51x128 – 0
BatchNorm-2 51x51x128 Offset: 1x1x128, 256 search for optimal solutions. It can use different types of probability
Scale: 1x1x128 models, but the most commonly used one is the Gaussian Process (GP).
Conv2D-6 49x49x128 Weights: 3x3x128, 147,584 GP is used as a surrogate model for objective functions in Bayesian
Bias: 1x1x128 optimization, as it represents a stochastic process that collects random
ReLu-6 49x49x128 0
variables. These variables are often indexed by space or time and their
–
Conv2D-7 46x46x128 Weights: 4x4x128x128, 262,272
Bias:1x1x128 distribution is described by a multivariate Gaussian distribution (Fraz­
ReLu-7 46x46x128 – 0 ier, 2018).GP can be expressed using Eq. (3) as a mean function (m(x))
MaxPool2D-3 9x9x128 0
–
and covariance (k(x, x′)). Eq. (4) shows how the f function maps inputs x
BatchNorm-3 9x9x128 Offset: 1x1x128, 256
Scale: 1x1x128
to outputs y. Eq. (5) models the noise (∊) with a Gaussian distribution
FC-1 1x1x1000 Weights: 1x1x1000, 10,369,000 (Bishnoi et al., 2021). The GP is updated by combining the newly ob­
Bias:1000x1 tained sample point (y) with the previously acquired samples.
FC-2 1x1x4 Weights: 4x1000, 4004
Bias:4x1 f (x) GP(m(x), k(x, x′) ) (3)
SoftMax 1x1x4 – 0
Classification 1x1x4 – 0 y = f (x) + ∊ (4)
Output
Number of total learnable parameters 10,912,972 ( )
∊ N 0, σ 2
(5)

technique. 4. Experimental results

x* = argmaxf (x), x ∈ X
Eq. (1) is used to find the best value of a function called f(x). It does this
by finding the value of x* that gives the best result for the function. The
× variable can include hyperparameters that can be real numbers,
(1)
This section presents the performance metrics, parameter optimiza­
tion results, results of proposed model, comparison results with pre-
trained CNN models, time complexity of models and comparison with
previous works. MATLAB 2021b software was used to implement the

Table 6
Results of hyperparameter optimization (1).
ML Algorithm Hyperparameters CNN Models
Darknet19 Darknet53 Densenet201 Efficientnetb0 Inceptionv3

DT Max. Number of Splits 190 256 427 641 263

Split Criterion MDR Twoing rule MDR MDR MDR
Total Elapsed Time(s) 259.37 240.54 249.54 240.15 243.26

NB Distribution Name Kernel Kernel Kernel Kernel Kernel

Kernel Type Gaussian Gaussian Gaussian Gaussian Gaussian
Total Elapsed Time(s) 14,256 14,416 14,293 13,581 14,738

KNN Number of Neighbors 1 1 2 1 1

Distance Metric Correlation Cosine Correlation Spearman Spearman
Distance Weight Inverse Inverse Inverse Inverse Inverse
Total Elapsed Time(s) 366.33 352.12 463.32 1143.5 398.98

SVM Kernel Type Quadratic Quadratic Quadratic Quadratic Quadratic

Box Constraint Level 173,4434 344,7146 0,0010016 999,6984 3,581
Multiclass Method One-vs-All One-vs-All One-vs-All One-vs-All One-vs-All
Total Elapsed Time(s) 17,499 14,495 14,132 15,677 22,218

8
M. Celik and O. Inik Expert Systems With Applications 238 (2024) 122159

Table 7
Results of hyperparameter optimization (2).
ML Algorithm Hyperparameters CNN Models
Nasnetmobile Resnet50 Resnet101 Xception Proposed

DT Max. Number of Splits 316 441 347 529 510

Split Criterion MDR MDR MDR MDR MDR
Total Elapsed Time(s) 258.35 245.99 251.29 255.5 278.49

NB Distribution Name Kernel Kernel Kernel Kernel Kernel

Kernel Type Gaussian Gaussian Gaussian Gaussian Gaussian
Total Elapsed Time(s) 14,259 16,958 14,141 14,931 14,557

KNN Number of Neighbors 1 1 1 1 1

Distance Metric Cosine Spearman Correlation Spearman City block
Distance Weight Squared inverse Inverse Squared inverse Inverse Inverse
Total Elapsed Time(s) 657.05 780.39 1009.6 394.36 593.71

SVM Kernel Type Quadratic Quadratic Gaussian Quadratic Linear

Box Constraint Level 0.0010033 1.0010013 953.0343 0.0010028 27.2379
Multiclass Method One-vs-All One-vs-One One-vs-All One-vs-All One-vs-One
Total Elapsed Time(s) 15,856 13,077 16,411 18,863 9665,8

deep learning and machine learning libraries in the experimental

studies. The computer used for the experiments had an Intel(R) Core
(TM) i5-8400 CPU @ 2.80 GHz (6 CPUs), 2.8 GHz, 16 GB RAM, and a
GPU NVIDIA GeForce GTX 1080 Ti with 11 GB memory.
4.1. Performance metrics
Various metrics are used to evaluate the performence of ML and DL
models. Different learning tasks may require different performance
metrics to be emphasized (Shaukat et al., 2020). A confusion matrix is
one way to formally present the performance of a learning model. A
confusion matrix, also known as an error matrix, is a table summarizing
the performance of a classification or prediction model, and a number of
classification performance metrics can be defined based on the confu­
sion matrix (Deng et al., 2016). Below are the commonly used metrics
that were selected as performance metrics for this study.
Accuracy can be defined as the ratio of correctly classified instances
to the total number of predictions (Eq. (6). This is a key performance
metric that measures how accurate the classification model produces
(Deng et al., 2016). A high accuracy value reflects the accuracy of the
classification model and therefore a higher accuracy value is preferred
(Shaukat et al., 2020).
TP + TN
Accuracy = (6)
TP + TN + FP + FN
Precision (Eq. (7) is a measure of accuracy in particular for accurately
predicting a class (Deng et al., 2016).
TP
Precision = (7)
TP + FP
2TP
F1 − score = (9)
2TP + FP + FN
True Positive (TP) refers to the number of positive samples correctly
classified by the model. True Negative (TN) is the number of instances
that a model classifies as negative are actually negative. That is, TN is
the number of instances in the negative class that are correctly classified.
False Positive (FP) refers to the number of negative instances that are
incorrectly classified as positive by the model. False Negative (FN) refers
to the number of positive samples misclassified as negative by the model
(Shaukat et al., 2020). In addition to these metrics, the complexity
analysis of the models is also included in Section 4.5 to compare model
performances.
4.2. Hyperparameter optimization results
The previous section provided a detailed discussion on the Bayesian
optimization of hyperparameters. In this section, the optimization out­
comes are presented in Tables 6 and 7. As evident in these tables, each
ML algorithm has distinct hyperparameters, and these values were ob­
tained through 20 iterations. However, the total computation times
varied across all algorithms. Notably, the DT algorithm was the fastest,
concluding the optimization process in 240.15 s for the Efficientnetb0
CNN model and 278.49 s for the proposed CNN model. Nonetheless, as
indicated in Table 8, DT had lower accuracy rates than other algorithms
in most cases. On the other hand, SVM algorithm was the slowest for
most runs, requiring 22,218 s for the Inceptionv3 CNN model and
9665.8 s for the proposed CNN model. However, SVM demonstrated the
highest classification accuracy among all other algorithms.

Recall (Eq. (8) is a measure of the ability of a prediction model to
accurately detect instances of a given class from a dataset (Deng et al.,
2016)Recall indicates how complete the detection of a class is.
TP
Recall = (8)
TP + FN
The F1-score is a metric commonly used to evaluate the performance of
machine learning or deep learning models. It is calculated by combining
the precision and recall of the model (Eq. (9). The F1-score can be
particularly useful when the class distribution is unbalanced (i.e., when
the number of samples in one class is significantly different from
another) and the user is seeking a trade-off between precision and recall.
A higher F1-score indicates that the model performs better than other
models (Shaukat et al., 2020).
4.3. Results of proposed CNN model
The proposed CNN model was trained for 20 epochs using a mini-
batch size of 64 and 1740 iterations, taking 56 min and 50 s. The Sto­
chastic Gradient Descent with Momentum (SGDM) solver was used with
an initial learning rate of 0.001. Fig. 5 displays the training and vali­
dation graphs for the proposed CNN model, depicting the loss values and
accuracy. The training accuracy was 100 % with a loss of 0, while the
validation accuracy was 95.66 % with a loss of 0.13 %. Additionally, the
confusion matrices for the proposed CNN and CNN-ML models are
presented in Fig. 6.
Fig. 6 shows that the CNN-KNN architecture had the lowest
misclassification error rate among all the models. The misclassification
errors for glioma, meningioma, tumor, and pituitary were 9, 20, 4, and

9
M. Celik and O. Inik Expert Systems With Applications 238 (2024) 122159

Table 8 4.4. Comparison results with state-of-the-art CNN models

Performance metrics obtained from confusion matrices.
Model Method Accuracy (%) Precision Recall F1-Score Brain MRI image classification was performed using both CNN and
ML algorithms. The ML algorithms utilized features derived from State-
Darknet19 CNN 92.95 0.93 0.93 0.93
DT 76 0.75 0.76 0.76 of-the-art CNN models, and Bayesian optimization was applied during
NB 75.95 0.75 0.75 0.74 training to optimize the hyperparameters of ML algorithms to enchance
KNN 94.66 0.95 0.95 0.95 their classification accuracy. The hyperparameters obtained for each
SVM 95.58 0.96 0.96 0.96 model are shown in Tables 6 and 7, and were not altered during the
training and testing phases. Four ML techniques were employed for
Darknet53 CNN 93.8 0.94 0.94 0.94 classifying the features of each model. The results of performance met­
DT 76 0.75 0.76 0.75
rics obtained from the confusion matrix of all CNN models are presented
NB 78.42 0.78 0.78 0.77
KNN 95.66 0.96 0.96 0.96 in Table 8, which indicates that the features extracted from each CNN
SVM 96.87 0.96 0.96 0.96 model were accurately classified using ML techniques that employed
hyperparameters generated via Bayesian optimization. The proposed
Densenet201 CNN 93.16 0.93 0.93 0.93 CNN model achieved the highest classification accuracy of 95.66 %
DT 78.77 0.78 0.78 0.78 among the CNN models. However, the SVM emerged as the strongest ML
NB 80.84 0.80 0.80 0.80 method for classification, achieving the highest accuracy of 97.93 % by
KNN 96.72 0.97 0.97 0.97
extracting features from brain MRI images using EfficientNetb0 and
SVM 97.01 0.97 0.97 0.97
classifying them using the SVM. After feature extraction, the proposed
CNN model achieved the highest accuracy of 97.15 % when classified
Efficientnetb0 CNN 95.51 0.95 0.96 0.95
using the KNN algorithm. The EfficientNetB0-SVM structure had preci­
DT 82.3 0.82 0.82 0.82
NB 82.5 0.82 0.83 0.82 sion, recall, and F1-Score values of 0.98, 0.98, and 0.98, respectively,
KNN 97 0.97 0.97 0.97 which are displayed in Table 8 and are higher compared to those ob­
SVM 97.93 0.98 0.98 0.98 tained using alternative CNN models. The ROC curves of all CNN-ML
structures are illustrated in Fig. 9. The average accuracy values across
Inceptionv3 CNN 91.74 0.92 0.92 0.92 all CNN models were 93.56 %, 95.99 %, 95.46 %, 77.78 %, and 76.04 %
DT 72.65 0.72 0.72 0.72 for the CNN, SVM, KNN, NB, and DT classification methods, respec­
NB 77.35 0.77 0.77 0.76
tively, suggesting that the classification of the structure and overall
KNN 94.52 0.94 0.95 0.94
SVM 95.3 0.95 0.96 0.95 prediction are more accurate.
Fig. 7 displays the confusion matrices resulting from the classifica­
tion outcomes of all pre-trained CNN models. The confusion matrices
Nasnetmobile CNN 90.03 0.9 0.9 0.9
DT 71.37 0.71 0.7 0.7 demonstrate that the no tumor class is the best-predicted class for each
NB 73.72 0.73 0.73 0.72 model, while the worst-predicted class varies among the models. Across
KNN 93.09 0.93 0.93 0.93 all CNN models, meningioma is the worst-predicted class. Meningioma
SVM 95.01 0.95 0.95 0.95
is also the worst-predicted class for the CNN-DT, CNN-NB, CNN-KNN,
and CNN-SVM architectures. The worst-predicted class for the CNN-
Resnet50 CNN 95.09 0.95 0.95 0.95 NB structures varies between meningioma and pituitary. The
DT 80.48 0.80 0.80 0.80
Darknet19-NB, Darknet53-NB, Inceptionv3-NB, and Nasnetmobile-NB
NB 78.70 0.78 0.79 0.78
KNN 96.15 0.96 0.96 0.96 structures perform poorly in classifying the pituitary class, while the
SVM 96.30 0.96 0.97 0.96 other CNN-NB structures are not successful in classifying the meningi­
oma class.
Resnet101 CNN 95.44 0.95 0.95 0.95 Based on the confusion matrices, the different hybrid models dis­
DT 76.85 0.76 0.76 0.76 played misclassifications. In particular, the DarkNet19-SVM hybrid
NB 75.43 0.75 0.75 0.75 model misclassified 62 tumors, with 10 Gliomas, 32 Meningiomas, 4 No
KNN 94.09 0.94 0.94 0.94
tumor, and 16 Pituitary labels being misclassified. The DarkNet53-SVM,
SVM 95.87 0.96 0.96 0.96
DenseNet201-SVM, EfficientNetB0-SVM, InceptionV3-SVM,
NasNetMobile-SVM, ResNet50-SVM, ResNet101-SVM, Xception-SVM,
Xception CNN 92.24 0.92 0.92 0.92
and proposed CNN-KNN hybrid models also had varying numbers of
DT 74.29 0.74 0.74 0.74
NB 79.49 0.79 0.79 0.79 misclassifications. The DarkNet53-SVM hybrid model also had some
KNN 95.51 0.95 0.95 0.96 misclassifications with 7 Gliomas, 33 Meningiomas, 4 No tumor, and 14
SVM 96.23 0.96 0.96 0.96 Pituitary labels misclassified as tumors. The DenseNet201-SVM hybrid
model showed misclassifications with 8 Gliomas, 25 Meningiomas, 1 No
Proposed CNN CNN 95.66 0.96 0.96 0.96 tumor, and 8 Pituitary labels misclassified as tumors. The
DT 71.72 0.71 0.71 0.71 EfficientNetB0-SVM hybrid model misclassified 4 Gliomas, 20 Menin­
NB 75.57 0.75 0.75 0.75
giomas, 1 No tumor, and 4 Pituitary labels as tumors. The InceptionV3-
KNN 97.15 0.97 0.97 0.97
SVM 93.86 0.94 0.94 0.94 SVM hybrid model also had misclassifications with 16 Gliomas, 36
Meningiomas, 3 No tumor, and 11 Pituitary labels misclassified as tu­
mors. The NasNetMobile-SVM hybrid model misclassified 22 Gliomas,
7, respectively. The CNN architecture was the second-best model with 25 Meningiomas, 9 No tumor, and 14 Pituitary labels as tumors. The
misclassification errors for glioma, meningioma, tumor, and pituitary ResNet50-SVM hybrid model misclassified 6 Gliomas, 28 Meningiomas,
calculated at 14, 28, 9, and 10, respectively. This suggests that menin­ 5 No tumor, and 12 Pituitary labels as tumors. The ResNet101-SVM
giomas are the most difficult type of brain tumor to classify accurately. hybrid model misclassified 9 Gliomas, 34 Meningiomas, 3 No tumor,
The accuracy of the CNN model and CNN-ML structures were 95.66 %, and 12 Pituitary labels as tumors. The Xception-SVM hybrid model
71.72 %, 75.57 %, 97.15 %, and 93.86 for the CNN, DT, NB, KNN, and misclassified 9 Gliomas, 31 Meningiomas, 4 No tumor, and 9 Pituitary
SVM, respectively. labels as tumors. The proposed CNN-KNN hybrid model misclassified 9
Gliomas, 20 Meningiomas, 4 No tumor, and 7 Pituitary labels as tumors,

10
M. Celik and O. Inik
Fig. 5. Accuracy and loss curves for proposed CNN model.
Fig. 6. Confusion matrices obtained from proposed CNN model and ML algorithms. Glioma = 0, Meningioma = 1, No tumor = 2, Pituitary = 3. The vertical axis of
each confusion matrix represents the actual classes while the horizontal axis of each confusion matrix represents predicted classes.
according to the confusion matrix.
The AUC-ROC curve, a critical evaluation metric in ML and DL, is
essential for assessing the effectiveness of a classification task, particu­
larly for imbalanced datasets. It represents the accuracy of the model in
making predictions, with the ROC curve reflecting a probability curve
for different classes. The X-axis denotes the False Positive Rate (FPR),
while the Y-axis denotes the True Positive Rate (TPR). For the CNN-SVM
architectures, Figs. 8 and 9 illustrate the ROC curves and AUC values of
CNN-ML structures. The proposed CNN-KNN structure has AUC values
of 0.98, 0.97, 0.98, and 1, respectively. It is evident that all CNN-SVM
structures have AUC values of 1, indicating excellent predictive
accuracy.
4.5. Complexity of models
In addition to accuracy, the complexity of deep learning models
should also be considered. As such, this study has included graphs that
depict accuracy-number of parameters, accuracy-floating point opera­
tions per second (FLOPs), and model-time to better express the
complexity of the models.
The first of these, the model size and model accuracy diagram, which
11
Expert Systems With Applications 238 (2024) 122159
helps to identify the most efficient model, is shown in Fig. 10. This graph
is used to evaluate the size and performance of a model together. Size is
important as it determines the computational cost. This type of graph
has been used in studies (Tan and Le, 2019) where state-of-the-art
models are compared with different models. In the graph, the x-axis is
the number of parameters, and the y-axis is the accuracy. The models are
given in ascending order according to the number of parameters. Ac­
cording to this sequence, the graph was obtained by giving the accuracy
value of each model. By looking at this graph, one can be easily seen the
measure of the accuracy of the models according to the number of pa­
rameters. This graph is a tool to study the relationship between model
size and accuracy. In this aspect, EfficientNetB0 and NasNetMobile
models are remarkable in terms of their size, but when analyzed in terms
of model accuracy and model size, EfficientNetB0 and the proposed CNN
model attract more attention. Moreover, the performance of these
models is better than the other models on the used dataset, and the lower
model sizes provide the advantage of less computational cost.
Secondly, the time complexity of the models used for brain tumor
classification was examined. All classification models took a significant
amount of time, depending on the complexity and architectural design
of the model. The time was measured in minutes and Fig. 11 shows the
M. Celik and O. Inik
Fig. 7. Confusion matrices of models.
training times of the models. The InceptionV3-SVM model gave good
classification results despite having thse longest training time (370 min).
The proposed CNN-KNN model had the second-best classification results
and completed the classification process in the shortest time (67 min).
The training of ResNet50-SVM and the proposed CNN-SVM models took
218 min, which is the shortest time for CNN-SVM classifiers. The
EfficientNetB0-SVM model achieved the highest classification accuracy
12
Expert Systems With Applications 238 (2024) 122159
and completed the training time in 261 min. In addition, EfficientNetB0-
KNN has both a low training time (79.06 min) and a high level of ac­
curacy. The DenseNet201-SVM model performed well in terms of both
training time and classification accuracy, with a training time of 236
min and ranking third in terms of performance. The Xception-SVM
model was the second model to complete training in over 300 min
(314 min). The ResNet101-SVM model lagged behind the ResNet50-
M. Celik and O. Inik Expert Systems With Applications 238 (2024) 122159

Fig. 8. ROC curves and AUC values of CNN-ML structures (a).

SVM model in terms of both training time and classification performance
and took 274 min to train. Finally, NasNetMobile-SVM has a training
time of 244 min, DarkNet53-SVM 242 min, and DarkNet19-SVM 292
min. However, it can be seen from the results that as the depth of the
network increases, the computational complexity, and thus the training
time, increases, which ultimately affects the efficiency of the network. In
addition, this study concludes that EfficientNetB0 and the proposed CNN
models are the best approach for classifying brain tumors.
Finally, the accuracy ratios of the models based on the FLOPs value,
which is an important parameter expressing computational complexity,

13
M. Celik and O. Inik Expert Systems With Applications 238 (2024) 122159

Fig. 9. ROC curves and AUC values of CNN-ML structures (b).

are presented in Fig. 12. The FLOPs value of a model is related to the
amount of GPU hardware resources that the model uses. A high FLOPs
value indicates a high computational cost, which can be a disadvantage
for the model. Models with high FLOPs may not be suitable for running
on lower hardware devices such as mobile devices. Therefore, it is
advantageous to develop models with lower FLOPs while achieving high
accuracy in model design. Fig. 12 shows that although the model with
the lowest FLOPs is NasNetMobile, its accuracy is low. On the other
hand, even though the FLOPs value of the proposed CNN model is
relatively high, its accuracy is still higher than that of the NasNetMobile

14
M. Celik and O. Inik Expert Systems With Applications 238 (2024) 122159

Fig. 10. Model size and model accuracy comparison of DL models used in the proposed method.

model. Looking at all the models, it can be seen that the FLOPs/accuracy
value of EfficientNetB0 is higher.

4.6. Comparison with previous studies

Table 9 shows the comparison of the proposed method with the

existing studies on the classification of brain MRI images. Specifically,
the proposed method is compared with existing ML and DL methods.
Table 9 presents a detailed comparison of DL and ML models for brain
tumor classification. The table includes the accuracy values, which is the
most commonly used performance measure in all the studies. The first
column includes previous studies, while the second column describes
the method used in each study. This can be a model proposed by the
authors or the use of state-of-the-art models. The third column describes
the dataset used in each study. The data set is emphasized as it is an
indicator that significantly affects the results. In this study, three
Fig. 11. Training time of models.
different data sets were combined and contained a significant amount of
data. In some studies, the healthy brain class was not taken into account
when performing the classification task, but in this study, this class was
also taken into account. In addition, since data augmentation also has an
impact on classification accuracy, the presence or absence of data
augmentation was also taken into account in the table. To compare and
evaluate the best available approaches, the best method and the best

Fig. 12. Accuracy comparison of DL models used in the proposed method according to FLOPs value.

15
M. Celik and O. Inik Expert Systems With Applications 238 (2024) 122159

Table 9
Comparison of the proposed method with previous studies.
Referance Model Dataset Number of Data Best Model Accuracy
Classes Augmentation

(Díaz-Pernas et al., 2021) Multi Scale CNN Figshare 3 Yes – 0.9730

(Sultan et al., 2019) CNN Figshare 3 Yes – 0.9613
(Ullah et al., 2022a) TL SARTAJ 3 Yes InceptionResNetV2 0.9891
(Deepak and Ameer, CNN-SVM Figshare 3 Yes CNN-SVM 0.9582
2021)
(Nayak et al., 2022) CNN Figshare 3 Yes Dense Efficient-Net 0.9997
(Alanazi et al., 2022) TL-CNN Figshare, SARTAJ, Br35H 3 Yes Proposed TL-CNN 0.9575
(Wahlang et al., 2022) LeNet Inspired Model Figshare, Brainweb, Radiopedia 3 No – 0.8800
combination
(Raza et al., 2022) TL- DeepTumorNet Figshare 3 No DeepTumorNet 0.9967
(Maqsood et al., 2022) Hybrid MobileNetV2 -M− SVM Figshare and BraTS 2018 3 No Hybrid MobileNetV2 0.9892
(Amran et al., 2022) Hybrid GoogLeNet and TL Br35H 3 Yes Hybrid GoogLeNet 0.9910
(Ullah et al., 2022b) TumorResNet and TL Abhranta Panigrahi 2 Yes TumorResNet 0.9933
(Deepak and Ameer, CNN-SVM, CNN-KNN Figshare 3 Yes CNN-KNN 0.9800
2019)
(Latif et al., 2022) CNN-ML BraTS 2018 (Glioma) 3 No CNN-SVM 0.9616
(Gómez-Guzmán et al., CNN with TL Figshare, Br35H and SARTAJ 4 Yes Proposed Generic 0.8105
2023) combination CNN
Inception V3 0.9712
Proposed Method State of the art CNN- Figshare, Br35H and SARTAJ 4 No EfficientNetB0-SVM 0.9793
optimized ML combination
Proposed Method Proposed CNN-optimized ML Figshare, Br35H and SARTAJ 4 No CNN-KNN 0.9715
combination

result for each study are included in the table. According to Table 9,
EfficientNetB0-SVM and the proposed CNN-KNN models outperform
some state-of-the-art models. These methods are capable of extracting
stronger and more distinguishable deep features and give the best results
for classification. Moreover, since the dataset used is balanced, there is a
sufficient number of images to train the network. In contrast, the data­
sets used in some studies have only two classes and not enough data for
classification. Therefore, the dataset used in this study is more complex
in terms of the number of images and classes and is a more difficult
classification problem than others. When Table 9 is analyzed, it was
observed that the highest classification accuracy value was obtained for
the dataset utilized in this study. The second-highest classification ac­
curacy was achieved by the proposed CNN method. It is noteworthy that
while state-of-the-art models have been trained on millions of images
previously, the proposed CNN model was trained solely on this dataset.
This outcome implies that the proposed model can be utilized effectively
for the classification of brain MRI images. Furthermore, despite not
having the same datasets, the proposed CNN has been observed to
achieve superior outcomes compared to studies that perform triple
classification to address the same problem.
5. Discussion
The task of classifying brain tumors is a challenging and time-
consuming process. Radiologists typically use brain MRI images to di­
agnose and categorize malignant tumors, but the possibility of human
error remains. To overcome this issue, DL + ML algorithms have been
employed to accurately classify brain tumors, as well as other internal
abnormalities. In this filed of study, both ML algorithms and DL models
are commonly used. The use of DL is preferred for its ability to directly
extract features and classify outputs from images via DNN. However,
this approach demands large datasets and a considerable number of
parameters in the deep network. In contrast, traditional ML algorithms
only accept specific image features as inputs, which can lead to time-
consuming preprocessing. As a result, researchers have focused on
combining DL and ML techniques to enhance classification accuracy.
In this section, the proposed model is assessed and compared to
existing state-of-the-art models, acknowledging that direct comparisons
may be challenging due to variations in datasets, analytical methods,
and simulation settings. CNN-based models(Díaz-Pernas et al., 2021;
Nayak et al., 2022; Sultan et al., 2019) have been proposed as an
effective approach for comprehensive learning through the isolation of
handcrafted feature extraction components. These models have mainly
utilized TL for training. Furthermore, various studies (Deepak and
Ameer, 2019, 2021; Latif et al., 2022) have been conducted wherein the
features derived from CNN models are classified using conventional ML
algorithms. Previous studies have certain limitations, including the use
of default hyperparameters for traditional ML methods, which can have
a considerable impact on classification accuracy. This paper introduces a
novel approach to brain tumor classification by presenting a newly
developed CNN model that is trained from scratch, without relying on
transfer learning techniques. The proposed methodology offers a hybrid
architecture that integrates both DL and ML methods and is different
from previous research on brain tumor classification. Specifically, the
proposed method adopts a novel hybrid approach that combines DL and
ML techniques, and includes feature extraction to improve classification
accuracy. The novelty of the proposed method lies in feeding the
extracted features into ML algorithms that are optimized using a
Bayesian optimization algorithm to improve the classification accuracy.
By leveraging the strengths of DL and ML methods, the proposed
approach achieves higher accuracy in the classification of brain tumors.
The hybrid model is proven to be efficient, rapid, and reliable, producing
promising results both through direct use of the CNN model and through
ML algorithms utilizing features extracted from the CNN model. Overall,
the proposed method offers several benefits over existing state-of-the-art
models, including its innovative training approach, the elimination of
the need for a separate feature extraction engine, and the use of a hybrid
model that combines CNN and ML techniques.
Based on the findings presented in Figs. 7, 8, and 9, as well as Ta­
bles 8 and 9, it is evident that the proposed CNN and CNN-ML structures
exhibit exceptional performance in classifying brain tumors. The results
indicate that the proposed CNN-KNN structure outperforms other CNN-
ML structures, except for the EfficientnetB0-SVM structure, which
demonstrated high accuracy in image classification. Furthermore, the
proposed CNN model achieved the highest accuracy rate among stand­
alone CNN models and surpassed state-of-the-art CNN models in terms
of accuracy when combined with the KNN algorithm. These results
highlight the potential of the proposed CNN and CNN-ML models for
accurate classification of brain tumors. In addition, in section 4.5, model
complexities are analyzed and evaluated in terms of time and compu­
tation. The classification accuracies and parameter numbers of the
models are also compared. Fıgure 10 shows the accuracy of different

16
M. Celik and O. Inik
models in relation to the number of parameters. It is useful for analyzing
the relationship between model size and accuracy. EfficientNetB0 and
NasNetMobile are notable for their small size, but when considering
both size and accuracy, EfficientNetB0 and the proposed CNN model
stand out. These models perform better than others on the dataset used
and their smaller size makes them computationally efficient. Fig. 11
shows training times of the models. The different models were evaluated
in terms of their training times and classification accuracy. The
InceptionV3-SVM model had the longest training time (370 min), but
gave good classification results. The proposed CNN-KNN model had the
second-best classification results and completed the classification pro­
cess in the shortest time (67 min). The EfficientNetB0-SVM model ach­
ieved the highest classification accuracy and completed training in 261
min. As the depth of the network increased, the training time also
increased, which ultimately affected the efficiency of the network. The
research findings suggest that for the classification of brain tumors, the
most effective approaches are the EfficientNetB0 and the proposed CNN
models. In addition, Fig. 12 provides information on the accuracy ratios
of various models based on the FLOPs value, a crucial parameter for
determining computational complexity. NasNetMobile exhibited the
lowest FLOPs value but with poor accuracy, whereas the proposed CNN
model demonstrated relatively high FLOPs value and still achieved
higher accuracy compared to the NasNetMobile model. Notably, the
EfficientNetB0 model demonstrated the highest FLOPs/accuracy ratio
compared to all the models evaluated.
6. Conclusion
This study demonstrates the potential of state-of-the-art CNN models
and a novel CNN model in accurately classifying brain tumours from
brain MRI images. ML algorithms, SVM, KNN, DT and NB, optimised
with Bayesian optimisation algorithm, were efficiently used to classify
features extracted from the models. Among the models tested, the
EfficientNetB0-SVM structure achieved the highest classification accu­
racy of 97.93 %. While SVM outperformed the other three ML algo­
rithms, the proposed CNN model achieved the highest accuracy of 97.15
% when paired with the KNN algorithm. Furthermore, the ten different
CNN models built on SVM had an average accuracy of 95.99 %. Effi­
cientNetB0 and the proposed CNN model performed better than other
models in terms of accuracy and computational efficiency, making them
the most suitable models for classifying brain tumors. It is worth noting
that the proposed CNN-KNN model completed the classification process
in the shortest time of 67 min, while achieving the second-best classi­
fication results. The proposed CNN model has a relatively high FLOPs
value but still achieved higher accuracy than the NasNetMobile model.
EfficientNetB0 had the highest FLOPs/accuracy ratio among all the
models, indicating its superiority in terms of computational complexity
and accuracy.
Overall, the findings of this study highlight the reliability and po­
tential of the proposed method in accurately classifying brain tumors,
which can lead to better patient outcomes and more effective treatment.
CRediT authorship contribution statement
Muhammed Celik: Methodology, Conceptualization, Data curation,
Software, Visualization, Writing – original draft. Ozkan Inik: Method­
ology, Supervision, Conceptualization, Writing – review & editing.
Declaration of Competing Interest
The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to influence
the work reported in this paper.
17
Expert Systems With Applications 238 (2024) 122159
Data availability
Data will be made available on request.
References
Aamir, M., Rahman, Z., Dayo, Z. A., Abro, W. A., Uddin, M. I., Khan, I., Imran, A. S.,
Ali, Z., Ishfaq, M., Guan, Y., & Hu, Z. (2022). A deep learning approach for brain
tumor classification using MRI images. Computers and Electrical Engineering, 101,
Article 108105. https://doi.org/10.1016/J.COMPELECENG.2022.108105
Abd-Ellah, M. K., Awad, A. I., Khalaf, A. A. M., & Hamed, H. F. A. (2018). Two-phase
multi-model automatic brain tumour diagnosis system from magnetic resonance
images using convolutional neural networks. Eurasip Journal on Image and Video
Processing, 2018(1), 1–10. https://doi.org/10.1186/S13640-018-0332-4/TABLES/7
Alam, T. M., Shaukat, K., Khan, W. A., Hameed, I. A., Almuqren, L. A., Raza, M. A.,
Aslam, M., & Luo, S. (2022). An Efficient Deep Learning-Based Skin Cancer Classifier
for an Imbalanced Dataset. Diagnostics 2022, Vol. 12, Page 2115, 12(9), 2115.
https://doi.org/10.3390/DIAGNOSTICS12092115.
Alanazi, M. F., Ali, M. U., Hussain, S. J., Zafar, A., Mohatram, M., Irfan, M., Alruwaili, R.,
Alruwaili, M., Ali, N. H., & Albarrak, A. M. (2022). Brain Tumor/Mass Classification
Framework Using Magnetic-Resonance-Imaging-Based Isolated and Developed
Transfer Deep-Learning Model. Sensors 2022, Vol. 22, Page 372, 22(1), 372. https://
doi.org/10.3390/S22010372.
Al-Badarneh, A., Najadat, H., & Alraziqi, A. M. (2012). A classifier to detect tumor
disease in MRI brain images. Proceedings of the 2012 IEEE/ACM International
Conference on Advances in Social Networks Analysis and Mining, ASONAM 2012,
784–787. https://doi.org/10.1109/ASONAM.2012.142.
Alharbi, F., Luo, S., Zhang, H., Shaukat, K., Yang, G., Wheeler, C. A., & Chen, Z. (2023). A
Brief Review of Acoustic and Vibration Signal-Based Fault Detection for Belt
Conveyor Idlers Using Machine Learning Models. Sensors 2023, Vol. 23, Page 1902,
23(4), 1902. https://doi.org/10.3390/S23041902.
Ali, Z., Hayat, M. F., Shaukat, K., Alam, T. M., Hameed, I. A., Luo, S., Basheer, S., Ayadi,
M., & Ksibi, A. (2022). A Proposed Framework for Early Prediction of
Schistosomiasis. Diagnostics 2022, Vol. 12, Page 3138, 12(12), 3138. https://doi.org/
10.3390/DIAGNOSTICS12123138.
Amran, G. A., Alsharam, M. S., Blajam, A. O. A., Hasan, A. A., Alfaifi, M. Y., Amran, M.
H., Gumaei, A., & Eldin, S. M. (2022). Brain Tumor Classification and Detection
Using Hybrid Deep Tumor Network. Electronics 2022, Vol. 11, Page 3457, 11(21),
3457. https://doi.org/10.3390/ELECTRONICS11213457.
Ananda Kumar, K. S., Prasad, A. Y., & Metan, J. (2022). A hybrid deep CNN-Cov-19-Res-
Net Transfer learning architype for an enhanced Brain tumor Detection and
Classification scheme in medical image processing. Biomedical Signal Processing and
Control, 76, Article 103631. https://doi.org/10.1016/J.BSPC.2022.103631
Aslan, M. F., Sabanci, K., Durdu, A., & Unlersen, M. F. (2022). COVID-19 diagnosis using
state-of-the-art CNN architecture features and Bayesian Optimization. Computers in
Biology and Medicine, 142, Article 105244. https://doi.org/10.1016/J.
COMPBIOMED.2022.105244
Aurna, N. F., Yousuf, M. A., Taher, K. A., Azad, A. K. M., & Moni, M. A. (2022).
A classification of MRI brain tumor based on two stage feature level ensemble of
deep CNN models. Computers in Biology and Medicine, 146, Article 105539. https://
doi.org/10.1016/J.COMPBIOMED.2022.105539
Badža, M. M., & Barjaktarović, M. C. (2020). Classification of Brain Tumors from MRI
Images Using a Convolutional Neural Network. Applied Sciences 2020, Vol. 10, Page
1999, 10(6), 1999. https://doi.org/10.3390/APP10061999.
Bhupendra, Moses, K., Miglani, A., & Kumar Kankar, P. (2022). Deep CNN-based damage
classification of milled rice grains using a high-magnification image dataset.
Computers and Electronics in Agriculture, 195, 106811. https://doi.org/10.1016/J.
COMPAG.2022.106811.
Bhuvaji, S., Kadam, A., Bhumkar, P., & Dedge, S. (2020). Brain Tumor Classification
(MRI) | Kaggle. https://doi.org/10.34740/kaggle/dsv/1183165.
Bishnoi, S., Ravinder, R., Grover, H. S., Kodamana, H., & Krishnan, N. M. A. (2021).
Scalable Gaussian processes for predicting the optical, physical, thermal, and
mechanical properties of inorganic glasses with large datasets. Materials Advances, 2
(1), 477–487. https://doi.org/10.1039/D0MA00764A
Cases | radiopaedia.org. (n.d.). Radiopaedia. Retrieved March 31, 2023, from https://ra
diopaedia.org/cases.
Chattopadhyay, A., & Maitra, M. (2022). MRI-based brain tumour image detection using
CNN based deep learning method. Neuroscience Informatics, 2(4), Article 100060.
https://doi.org/10.1016/J.NEURI.2022.100060
Cheng, J. (2017). Brain Tumor MRI Dataset. https://figshare.com/articles/dataset/
brain_tumor_dataset/1512427.
Cheng, J., Huang, W., Cao, S., Yang, R., Yang, W., Yun, Z., Wang, Z., & Feng, Q. (2015).
Enhanced Performance of Brain Tumor Classification via Tumor Region
Augmentation and Partition. PLoS One1, 10(10), e0140381.
Chollet, F. (2016). Xception: Deep Learning with Depthwise Separable Convolutions.
Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR
2017, 2017-January, 1800–1807. https://doi.org/10.48550/arxiv.1610.02357.
Collins, D. L., Zijdenbos, A. P., Kollokian, V., Sled, J. G., Kabani, N. J., Holmes, C. J., &
Evans, A. C. (1998). Design and construction of a realistic digital brain phantom.
IEEE Transactions on Medical Imaging, 17(3), 463–468. https://doi.org/10.1109/
42.712135
Das, S., Aranya, O. F. M. R. R., & Labiba, N. N. (2019). Brain Tumor Classification Using
Convolutional Neural Network. 1st International Conference on Advances in Science,
M. Celik and O. Inik
Engineering and Robotics Technology 2019, ICASERT 2019. https://doi.org/10.1109/
ICASERT.2019.8934603.
Deepak, S., & Ameer, P. M. (2019). Brain tumor classification using deep CNN features
via transfer learning. Computers in Biology and Medicine, 111, Article 103345.
https://doi.org/10.1016/J.COMPBIOMED.2019.103345
Deepak, S., & Ameer, P. M. (2021). Automated Categorization of Brain Tumor from MRI
Using CNN features and SVM. Journal of Ambient Intelligence and Humanized
Computing, 12(8), 8357–8369. https://doi.org/10.1007/S12652-020-02568-W/
FIGURES/7
Deng, J., Dong, W., Socher, R., Li, L.-J., Kai Li, & Li Fei-Fei. (2010). ImageNet: A large-
scale hierarchical image database. 248–255. https://doi.org/10.1109/
CVPR.2009.5206848.
Deng, X., Liu, Q., Deng, Y., & Mahadevan, S. (2016). An improved method to construct
basic probability assignment based on the confusion matrix for classification
problem. Information Sciences, 340–341, 250–261. https://doi.org/10.1016/J.
INS.2016.01.033
Devnath, L., Luo, S., Summons, P., Wang, D., Shaukat, K., Hameed, I. A., & Alrayes, F. S.
(2022). Deep Ensemble Learning for the Automatic Detection of Pneumoconiosis in
Coal Worker’s Chest X-ray Radiography. Journal of Clinical Medicine 2022, Vol. 11,
Page 5342, 11(18), 5342. https://doi.org/10.3390/JCM11185342.
Díaz-Pernas, F. J., Martínez-Zarzuela, M., González-Ortega, D., & Antón-Rodríguez, M.
(2021). A Deep Learning Approach for Brain Tumor Classification and Segmentation
Using a Multiscale Convolutional Neural Network. Healthcare 2021, Vol. 9, Page 153,
9(2), 153. https://doi.org/10.3390/HEALTHCARE9020153.
El-Dahshan, E. S. A., Hosny, T., & Salem, A. B. M. (2010). Hybrid intelligent techniques
for MRI brain images classification. Digital Signal Processing, 20(2), 433–441. https://
doi.org/10.1016/J.DSP.2009.07.002
Frazier, P. I. (2018). A Tutorial on Bayesian Optimization. https://doi.org/10.48550/
arxiv.1807.02811.
Gómez-Guzmán, M. A., Jiménez-Beristaín, L., García-Guerrero, E. E., López-Bonilla, O.
R., Tamayo-Perez, U. J., Esqueda-Elizondo, J. J., Palomino-Vizcaino, K., & Inzunza-
González, E. (2023). Classifying Brain Tumors on Magnetic Resonance Imaging by
Using Convolutional Neural Networks. Electronics 2023, Vol. 12, Page 955, 12(4),
955. https://doi.org/10.3390/ELECTRONICS12040955.
Hamada, A. (2020). Br35H :: Brain Tumor Detection 2020 | Kaggle. https://www.kaggle.
com/datasets/ahmedhamada0/brain-tumor-detection?select=no.
He, K., Zhang, X., Ren, S., & Sun, J. (2015). Deep Residual Learning for Image
Recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision
and Pattern Recognition, 2016-December, 770–778. https://doi.org/10.48550/
arxiv.1512.03385.
Hemanth, G., Janardhan, M., & Sujihelen, L. (2019). Design and implementing brain
tumor detection using machine learning approach. Proceedings of the International
Conference on Trends in Electronics and Informatics, ICOEI 2019, 2019-April,
1289–1294. https://doi.org/10.1109/ICOEI.2019.8862553.
Huang, G., Liu, Z., van der Maaten, L., & Weinberger, K. Q. (2016). Densely Connected
Convolutional Networks. Proceedings - 30th IEEE Conference on Computer Vision and
Pattern Recognition, CVPR 2017, 2017-January, 2261–2269. https://doi.org/
10.48550/arxiv.1608.06993.
Inik, O. (2023). CNN hyper-parameter optimization for environmental sound
classification. Applied Acoustics, 202, Article 109168. https://doi.org/10.1016/J.
APACOUST.2022.109168
Jibon, F. A., Khandaker, M. U., Miraz, M. H., Thakur, H., Rabby, F., Tamam, N.,
Sulieman, A., Itas, Y. S., & Osman, H. (2022). Cancerous and Non-Cancerous Brain
MRI Classification Method Based on Convolutional Neural Network and Log-Polar
Transformation. Healthcare 2022, Vol. 10, Page 1801, 10(9), 1801. https://doi.org/
10.3390/HEALTHCARE10091801.
Johnson, K. A., & Becker, J. A. (1999). The Whole Brain Atlas. https://www.med.
harvard.edu/AANLIB/.
Inik, O., & Ulker, E. (2022). Optimization of deep learning based segmentation method.
Soft Computing, 26(7), 3329–3344. https://doi.org/10.1007/S00500-021-06711-3/
TABLES/9
Ini˙k, O., Altıok, M., Ulker, E., & Kocer, B. (2021). MODE-CNN: A fast converging multi-
objective optimization algorithm for CNN-based models. Applied Soft Computing, 109,
107582. https://doi.org/10.1016/J.ASOC.2021.107582
Khan, H. A., Jue, W., Mushtaq, M., & Mushtaq, M. U. (2020). Brain tumor classification in
MRI image using convolutional neural network. Mathematical Biosciences and
Engineering : MBE, 17(5), 6203–6216. https://doi.org/10.3934/MBE.2020328
Kumar, M. R., Vekkot, S., Lalitha, S., Gupta, D., Govindraj, V. J., Shaukat, K., Alotaibi, Y.
A., & Zakariah, M. (2022). Dementia Detection from Speech Using Machine Learning
and Deep Learning Architectures. Sensors 2022, Vol. 22, Page 9311, 22(23), 9311.
https://doi.org/10.3390/S22239311.
Kumar, S., & Mankame, D. P. (2020). Optimization driven Deep Convolution Neural
Network for brain tumor classification. Biocybernetics and Biomedical Engineering, 40
(3), 1190–1204. https://doi.org/10.1016/J.BBE.2020.05.009
Latif, G., Brahim, G. Ben, Awang Iskandar, D. N. F., Bashar, A., & Alghazo, J. (2022).
Glioma Tumors’ Classification Using Deep-Neural-Network-Based Features with
SVM Classifier. Diagnostics, 12(4), 1018. https://doi.org/10.3390/
DIAGNOSTICS12041018
Maqsood, S., Damaševičius, R., & Maskeliūnas, R. (2022). Multi-Modal Brain Tumor
Detection Using Deep Neural Network and Multiclass SVM. Medicina 2022, Vol. 58,
Page 1090, 58(8), 1090. https://doi.org/10.3390/MEDICINA58081090.
Menze, B. H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., … Van
Leemput, K. (2015). The Multimodal Brain Tumor Image Segmentation Benchmark
18
Expert Systems With Applications 238 (2024) 122159
(BRATS). IEEE Transactions on Medical Imaging, 34(10), 1993–2024. https://doi.org/
10.1109/TMI.2014.2377694
Mohsen, H., El-Dahshan, E.-S.-A., El-Horbaty, E.-S.-M., & Salem, A.-B.-M. (2018).
Classification using deep learning neural networks for brain tumors. Future
Computing and Informatics Journal, 3(1), 68–71. https://doi.org/10.1016/j.
fcij.2017.12.001
Nayak, D. R., Padhy, N., Mallick, P. K., Zymbler, M., & Kumar, S. (2022). Brain Tumor
Classification Using Dense Efficient-Net. Axioms 2022, Vol. 11, Page 34, 11(1), 34.
https://doi.org/10.3390/AXIOMS11010034.
Nickparvar, M. (2021). Brain Tumor MRI Dataset | Kaggle. https://doi.org/10.34740/
kaggle/dsv/2645886.
Panigrahi, A. (2021). Brain_Tumor_Detection_MRI | Kaggle. https://www.kaggle.com/
datasets/abhranta/brain-tumor-detection-mri.
Raza, A., Ayub, H., Khan, J. A., Ahmad, I., Salama, A. S., Daradkeh, Y. I., Javeed, D.,
Rehman, A. U., & Hamam, H. (2022). A Hybrid Deep Learning-Based Approach for
Brain Tumor Classification. Electronics 2022, Vol. 11, Page 1146, 11(7), 1146.
https://doi.org/10.3390/ELECTRONICS11071146.
Redmon, J., & Farhadi, A. (2016). YOLO9000: Better, Faster, Stronger. Proceedings - 30th
IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, 2017-
January, 6517–6525. https://doi.org/10.1109/CVPR.2017.690.
Redmon, J., & Farhadi, A. (2018). YOLOv3: An Incremental Improvement. https://doi.
org/10.48550/arxiv.1804.02767.
Samee, N. A., Mahmoud, N. F., Atteia, G., Abdallah, H. A., Alabdulhafith, M., Al-
Gaashani, M. S. A. M., Ahmad, S., & Muthanna, M. S. A. (2022). Classification
Framework for Medical Diagnosis of Brain Tumor with an Effective Hybrid Transfer
Learning Model. Diagnostics 2022, Vol. 12, Page 2541, 12(10), 2541. https://doi.org/
10.3390/DIAGNOSTICS12102541.
Sharif, M. I., Li, J. P., Khan, M. A., & Saleem, M. A. (2020). Active deep neural network
features selection for segmentation and recognition of brain tumors using MRI
images. Pattern Recognition Letters, 129, 181–189. https://doi.org/10.1016/J.
PATREC.2019.11.019
Shaukat, K., Luo, S., & Varadharajan, V. (2023). A novel deep learning-based approach
for malware detection. Engineering Applications of Artificial Intelligence, 122, Article
106030. https://doi.org/10.1016/J.ENGAPPAI.2023.106030
Shaukat, K., Luo, S., Varadharajan, V., Hameed, I. A., & Xu, M. (2020). A Survey on
Machine Learning Techniques for Cyber Security in the Last Decade. IEEE Access, 8,
222310–222354. https://doi.org/10.1109/ACCESS.2020.3041951
Srinivas, C., Nandini, N. P., Zakariah, M., Alothaibi, Y. A., Shaukat, K., Partibane, B., &
Awal, H. (2022). Deep Transfer Learning Approaches in Performance Analysis of
Brain Tumor Classification Using MRI Images. Journal of Healthcare Engineering,
2022. https://doi.org/10.1155/2022/3264367
Sultan, H. H., Salem, N. M., & Al-Atabany, W. (2019). Multi-Classification of Brain Tumor
Images Using Deep Neural Network. IEEE Access, 7, 69215–69225. https://doi.org/
10.1109/ACCESS.2019.2919122
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2015). Rethinking the
Inception Architecture for Computer Vision. Proceedings of the IEEE Computer Society
Conference on Computer Vision and Pattern Recognition, 2016-December, 2818–2826.
https://doi.org/10.48550/arxiv.1512.00567.
Tan, M., & Le, Q. v. (2019). EfficientNet: Rethinking Model Scaling for Convolutional
Neural Networks. 36th International Conference on Machine Learning, ICML 2019,
2019-June, 10691–10700. https://doi.org/10.48550/arxiv.1905.11946.
Ullah, N., Khan, J. A., Khan, M. S., Khan, W., Hassan, I., Obayya, M., Negm, N., &
Salama, A. S. (2022). An Effective Approach to Detect and Identify Brain Tumors
Using Transfer Learning. Applied Sciences 2022, Vol. 12, Page 5645, 12(11), 5645.
https://doi.org/10.3390/APP12115645.
Ullah, N., Khan, M. S., Khan, J. A., Choi, A., & Anwar, M. S. (2022). A Robust End-to-End
Deep Learning-Based Approach for Effective and Reliable BTD Using MR Images.
Sensors 2022, Vol. 22, Page 7575, 22(19), 7575. https://doi.org/10.3390/
S22197575.
Wahlang, I., Maji, A. K., Saha, G., Chakrabarti, P., Jasinski, M., Leonowicz, Z., &
Jasinska, E. (2022). Brain Magnetic Resonance Imaging Classification Using Deep
Learning Architectures with Gender and Age. Sensors 2022, Vol. 22, Page 1766, 22
(5), 1766. https://doi.org/10.3390/S22051766.
Wu, J., Chen, X. Y., Zhang, H., Xiong, L. D., Lei, H., & Deng, S. H. (2019).
Hyperparameter Optimization for Machine Learning Models Based on Bayesian
Optimization. Journal of Electronic Science and Technology, 17(1), 26–40. https://doi.
org/10.11989/JEST.1674-862X.80904120.
Yazdan, S. A., Ahmad, R., Iqbal, N., Rizwan, A., Khan, A. N., & Kim, D. H. (2022). An
Efficient Multi-Scale Convolutional Neural Network Based Multi-Class Brain MRI
Classification for SaMD. Tomography 2022, Vol. 8, Pages 1905-1927, 8(4),
1905–1927. https://doi.org/10.3390/TOMOGRAPHY8040161.
Zacharaki, E. I., Wang, S., Chawla, S., Yoo, D. S., Wolf, R., Melhem, E. R., &
Davatzikos, C. (2009). Classification of brain tumor type and grade using MRI
texture and shape in a machine learning scheme. Magnetic Resonance in Medicine, 62
(6), 1609–1618. https://doi.org/10.1002/MRM.22147
Zhang, Y., Dong, Z., Wu, L., & Wang, S. (2011). A hybrid method for MRI brain image
classification. Expert Systems with Applications, 38(8), 10049–10053. https://doi.org/
10.1016/J.ESWA.2011.02.012
Zoph, B., Vasudevan, V., Shlens, J., & Le, Q. V. (2017). Learning Transferable
Architectures for Scalable Image Recognition. Proceedings of the IEEE Computer
Society Conference on Computer Vision and Pattern Recognition, 8697–8710. https://
doi.org/10.48550/arxiv.1707.07012.