Computational Biology and Chemistry 88 (2020) 107329

Contents lists available at ScienceDirect

Computational Biology and Chemistry

journal homepage: www.elsevier.com/locate/cbac

A deep learning approach based on convolutional LSTM for detecting T

diabetes
Motiur Rahman*, Dilshad Islam, Rokeya Jahan Mukti, Indrajit Saha
Department of Physical and Mathematical Sciences, Chattogram Veterinary and Animal Sciences University, Chattogram, Bangladesh

A R T I C LE I N FO A B S T R A C T

Keywords: Diabetes is a chronic disease that occurs when the pancreas does not generate sufficient insulin or the body
Diabetes detection cannot effectively utilize the produced insulin. If it remains unidentified and untreated, then it could be very
Feature selection deadliest. One can lead a healthy life with proper treatment if the presence of diabetes can be detected at an
Parameter optimization early stage. When the conventional process of detecting diabetes is tedious, there is a need of an automated
LSTM
system for identifying diabetes from the clinical and physical data. In this study, we developed a novel diabetes
Conv-LSTM
CNN
classifying model based on Convolutional Long Short-term Memory (Conv-LSTM) that was not applied yet in this
regard. We applied another three popular models such as Convolutional Neural Network (CNN), Traditional
LSTM (T-LSTM), and CNN-LSTM and compared the performance with our developed model over the Pima
Indians Diabetes Database (PIDD). Significant features were extracted from the dataset using Boruta algorithm
that returned glucose, BMI, insulin, blood pressure, and age as important features for classifying diabetes pa-
tients more accurately. We performed hyperparameter optimization using Grid Search algorithm in order to find
the optimal parameters for the applied models. Initial experiment by splitting the dataset into separate training
and testing sets, the Conv-LSTM-based model classified the diabetes patients with the highest accuracy of 91.38
%. In later, using cross-validation technique the Conv-LSTM model achieved the highest accuracy of 97.26 % and
outperformed the other three models along with the state-of-the-art models.

1. Introduction
Diabetes is one of the common deadliest and chronic diseases that
causes many complications if it remains untreated and unidentified.
Higher concentration of glucose than the normal level is presented in
blood due to diabetes. If it remains unidentified and untreated, then it
can lead to serious damages in eyes, kidneys, heart, blood vessels, and
nerves (Sneha and Gangil, 2019). Two types of diabetes are found
among the patients- type-1 and type-2 diabetes (Allam et al., 2011).
Normally the young people, mostly less than 30 years old, are affected
by type-1 diabetes. Patients with type-1 diabetes cannot be cured ef-
fectively with oral medications only while it requires insulin therapy to
treat it properly (Li et al., 2018). In contrast, type-2 diabetes occurs
commonly in middle-aged and older-aged people, which is not com-
pletely curable. Type-2 diabetes affected people can lead a normal life
by controlling lifestyle and regular checkups (Massaro et al., 2019).
The International Diabetes Federation conducted a study in 2019
that unveiled approximately 463 million adults are living with diabetes
and also projected that it will rise to 700 million by 2045. In addition,
one of every 6 six live births are affected by diabetes during pregnancy.
Despite a huge expenditure of USD 760 billion dollars in diabetes, one
in every two people with diabetes remain undiagnosed (Mhaskar et al.,
2017). Additionally, 374 million people are at escalating risk of af-
fecting diabetes. Development of living standards is mostly responsible
for this unwanted increase of diabetes patients.
A diabetes patient can lead a normal life by proper diagnosis and
treatment. There have different measures like A1c, random blood sugar,
fasting blood sugar, and oral glucose tolerance test, etc for identifying
diabetes (Ramachandran, 2020). Depending on the type of diabetes,
patients may check and record blood glucose concentration as many
four times a day or more if they are taking insulin. Diabetes identifi-
cation based on a single parameter may lead to misdiagnosis and mis-
lead decision making. High levels of vitamin E ingestion can falsely
increase A1c levels while vitamin B-9 and B-12 can falsely lower A1c
levels (Guo et al., 2010). Ingestion of vitamin C may decline A1c when
estimated by chromatography but levels may elevate when estimated
by electrophoresis (Clark et al., 2007). Hence, there is a need to com-
bine different parameters to diagnose diabetes effectively. Diagnosis
with the combination of glucose, BMI, diabetes pedigree function,
blood pressure, age, pregnancy, skin thickness, and insulin will be more

⁎
Corresponding author.
E-mail address: motiur@cvasu.ac.bd (M. Rahman).

https://doi.org/10.1016/j.compbiolchem.2020.107329
Received 25 April 2020; Received in revised form 29 May 2020; Accepted 2 July 2020
Available online 10 July 2020
1476-9271/ © 2020 Elsevier Ltd. All rights reserved.
M. Rahman, et al.
effective to identify and treat diabetes.
A huge volume of clinical data is generated by each healthcare that
can be processed to retrieve useful information. The development of
machine learning algorithms assists to process a huge volume of data
and extract the underlying data pattern that facilitates the decision-
making process (Shi et al., 2015). Knowledge from data can be gathered
when a machine learning model can be trained with the observed data.
Several machine learning techniques have been developed for pre-
dicting disease and discovering knowledge from medical data. A re-
current neural network unit called LSTM is widely used in different
prediction problems (Sun et al., 2018). The concept of memory in LSTM
helps to capture the underlying data pattern and long dependency
among data during the training. The actual LSTM faces a problem called
vanishing gradient that can be overcome by another architecture of
LSTM called Convolutional LSTM (Rahman and Adjeroh, 2019). An
artificial neural network has a strong predictive power that can be used
to model complex prediction problems. A decision tree is a supervised
machine learning algorithm that is widely used in data mining and
various predictive problems. Random forest generates many trees to
make a conclusion that also has a huge classification power. The pre-
diction accuracy of a machine learning technique highly depends on a
better selection of attributes (Fortino et al., 2014). Feature selection is a
process of shortening a large feature set to a subset that has more im-
pact on the outcome. A better subset of features facilitates to fight
against the curse of dimensionality, to avoid overfitting, to increase
generalizability, and to reduce the overall training times. Several fea-
ture selection algorithms are used to select the most relevant subset of
features. A feature ranking and selection technique named Boruta based
on a random forest algorithm is widely used for narrowing a large
feature set to a subset (Ali et al., 2019).
In this study, we employed Conv-LSTM, CNN, T-LSTM, and CNN-
LSTM for predicting diabetes. The novelty of this work is that nobody
has applied Conv-LSTM yet for diabetes prediction although it has
outperformed in various classification and prediction problems. We
utilized a random forest-based Boruta feature selection algorithm for
capturing the most significant features. Different comparative analyses
were conducted to find out where our model performs better and vice-
versa. Performance comparison of our model with other models depicts
that Conv-LSTM-based model achieved 97.26 % accuracy that is higher
than the other models. We have also tuned different parameters of our
model in order to find out the most suitable configuration when the
model is more accurate. We have performed hyperparameter optimi-
zation using grid search algorithm. Besides, we have also reported
where our model performs better and the reasons behind. We have
compared performance of our model with that of the state-of-the-art
models.
The rest of the paper is organized as follows- the related literatures
are reviewed in Section 2; Section 3 outlines the used dataset; the ap-
plied models to classify diabetes patients are depicted in Section 4;
Section 5 reviews the conducted experiments and architecture of the
model; results and discussion are narrated in Section 6; finally Section 7
draws the conclusion with some future recommendations.
2. Related research
Although a need of an automated diabetes classifying system is
immense, a significant number of researches have not been carried out
on diabetes prediction using machine learning algorithms. Even though
the accuracy of the so far completed works is quite good, still there is a
scope of doing research to improve the accuracy of diabetes patient
classification. Some recent works on diabetes prediction using machine
learning algorithms are described in this section.
Recently, various machine learning algorithms have shown a pro-
mising outcome in different medical research. Likewise, some machine
learning algorithms such as Decision Tree, Random Forest SVM, and
Naive Bayes are widely used for classifying and predicting diabetes
2
Computational Biology and Chemistry 88 (2020) 107329
patients by observing different attributes. A study was conducted using
the Pima Indians Diabetes dataset to predict diabetes at an early stage.
They used three machine-learning algorithms namely Decision Tree,
SVM and Naive Bayes classifier for detecting diabetes, and they found
that the Naive Bayes classifier outperformed the other two classifiers
with an accuracy of 76.30 % (Singh et al., 2018). The artificial neural
network puts a significant contribution to diabetes prediction as it has
strong predictive power. Quan et al. examined different algorithms
namely decision tree, random forest and neural network to predict
diabetes mellitus (Zou et al., 2018). They used data from a hospital in
Luzhou, China for training and testing the models, and used five-fold
cross-validation technique for evaluating the models. Their experi-
mental results indicated that the random forest algorithm achieved the
highest accuracy of 80.80 %. Training and testing a machine learning
model with optimized features returns better accuracy rather than with
un-optimized features. Sneha et al. selected significant features from
Pima Indians Diabetes dataset using a modified approach, and that
features were used for early prediction of diabetes with SVM, random
forest, naive Bayes, decision tree, and k-nearest neighbor classifiers
(Sneha and Gangil, 2019). They found the highest accuracy in terms of
diabetes prediction with the SVM classifier. The decision tree algorithm
was applied by Apoorva et al. to predict type-2 diabetes using the PID
dataset. Performance comparison with SVM classifier justified that the
decision tree forecasted type-2 diabetes accuratel (Apoorva et al.,
2020). Yukai et al. developed techniques for diabetes prediction from
follow-up data using SVM, decision tree, and ensemble learning model,
and found that the decision tree classifier returned a lower accuracy
than the other two (Li et al., 2018).
Another popular form of machine learning algorithm known as a
recurrent neural network (RNN) is widely used in various prediction
problems. A recurrent neural unit namely LSTM (long short-term
memory) resolves the problem of long-term dependency among data
and performs well in diabetes prediction also. Qingnan et al. proposed a
deep neural network-based model using LSTM and Bi-LSTM for diabetes
prediction, and performance comparison with ARIMA and SVR classi-
fiers showed that their proposed model outperformed the other two
models (Sun et al., 2018). A deep learning model based on solid
mathematical theory and domain knowledge was suggested to predict
blood glucose concentration from continuous glucose monitoring
system data 30 min ahead. Another study using recurrent neural net-
work (RNN) and data obtained from CGM devices was conducted to
predict type-1 diabetes. Performance comparison revealed that the RNN
based model performed better rather than the feed-forward neural
network prediction model (NNM). Swapna et al. developed a metho-
dology for the classification of diabetes and normal HRV signal by in-
corporating long short-term memory (LSTM), convolutional neural
network (CNN) and their combinations (Swapna et al., 2018). They
obtained performance improvement of 0.03 % and 0.06 % in CNN and
CNN-LSTM architecture respectively compared to their early work
based on SVM.
The aforementioned machine learning algorithms like the random
forest, SVM, decision tree, LSTM, ANN, CNN, RNN, ARIMA, and SVR
perform well in diabetes prediction. However, the random forest al-
gorithm does well in classification problems while it has limitations in
regression problems as it doesn't predict beyond the range in the
training data. The decision tree algorithm is unstable as a slight change
in data causes a larger change in the structure of the decision
(Kotsiantis, 2013). Besides, the predictive power of the decision tree
algorithm is inadequate for applying in regression problems. The sup-
port vector machine (SVM) algorithm does not perform well when the
dataset has more noise (Zhang, 2012). These three algorithms perform
well for classification problems while ANN, CNN, and LSTM algorithms
perform well for regression problems. Though the ANN, CNN, and
LSTM do well in predictive problems, they face a problem called van-
ishing gradient that makes it harder to capture the impact of the earliest
stages when the length of input sequence is increased (Yamashita et al.,
M. Rahman, et al. Computational Biology and Chemistry 88 (2020) 107329

2018; Agatonovic-Kustrin and Beresford, 2000; Staudemeyer and Table 2

Morris, 2019). This problem has a negative impact on the output. This Hyperparameter optimization of Conv-LSTM and T-LSTM.
problem can be overcome by using an architecture of LSTM called Conv-LSTM
Convolutional LSTM that consists of a memory cell along with four
gates. The Conv-LSTM has an extra connection with the previous Learning rate Batch size Hidden units Epoch Mean test score
memory cells that allows it to capture the effect of the previous input in 0.01 32 100 300 98.24
0.01 64 100 300 97.01
the current timestamp (Shi et al., 2015). The memory cell allows it to 0.03 32 100 300 96.84
capture the impact of the earliest stages during the training phase T-LSTM
(Rahman and Adjeroh, 2019). Besides, an additional connection to the 0.01 32 100 300 91.31
previous memory cell called peephole connection prevents from oc- 0.03 32 100 300 89.64
0.01 64 100 300 89.17
curring vanishing gradient problems. In this paper, we developed a
model based on the Convolutional LSTM to predict diabetes using se-
lective features. Here, the significant features were selected using a
gradient problems etc. that have adverse effects on performance. These
random forest algorithm-based algorithm namely Boruta.
limitations can be overcome by applying a variation of LSTM named as
Convolutional LSTM (Conv-LSTM) that allows to capture the impact of
3. Dataset the earliest given data. The vanishing gradient problem can be elimi-
nated by tuning the weight value during the training phase. In this
The performance of our Conv-LSTM based model was evaluated by study, we employed Conv-LSTM, T-LSTM, CNN, and CNN-LSTM over
using a popular dataset namely Pima Indians Diabetes Database (PIDD) the Pima Indian Diabetes dataset to observe the performance of these
(Kamble et al., 2016) obtained from the UCI machine learning re- models in terms of accurate diabetes patient classification.
pository. The dataset contains records of 768 female patients aged at
least 21 years among them 268 are diabetes positive and the rest are 4.1. Traditional LSTM
diabetes negative. The dataset has eight predictor variables like Preg-
nancies, Glucose, Blood Pressure, BMI, Skin Thickness, Insulin, Diabetes A variant of recurrent neural network called Long Short-term
Pedigree Function, and Age for diagnostically predicting whether a Memory (LSTM) consists of a memory cell and four gates namely forget
patient has diabetes or not and one target variable named as outcome. gate f , input gate i , control gate c and output gate o (Muzaffar and
During the early stage of pregnancy, blood glucose level may in- Afshari, 2019). It can extract and remember the underlying data pattern
crease that leads of diabetes-related complications. Another important that resolves the long-term data dependency among data occurred in
indicator of diabetes is glucose if it presents more in blood than the classical RNN algorithms. The LSTM comes in various forms such as
normal level. High glucose level in blood can increase blood pressure Traditional LSTM (T-LSTM), Peephole LSTM, and Convolutional LSTM
that is a significant symptom of diabetes. Being overweight greatly (Conv-LSTM) that have different architectures (Zhu et al., 2019). The
increases the risk of developing type 2 diabetes. The imbalance of in- diagram of T-LSTM as shown in Fig. 1 (Rahman and Siddiqui, 2019)
sulin is also an important exhibitor of diabetes. Since diabetes may indicates that it takes previous cell state ht − 1, current input vector x t ,
come from family, diabetes pedigree function provides useful data on and bias b as inputs and generates current memory content ct and
diabetes. There is evident of getting skin thicken who are suffering from current cell state ht as final outputs after performing some operations
insulin-dependent diabetes mellitus. The risk of type 2 diabetes in- over the previous memory content ct − 1. These above mentioned four
creases as people get older, especially after age 45. Therefore, we can gates control the content of the memory cell.
say that all the mentioned features are important for detecting diabetes. The forget gate generates a value in the rage of 0–1 that decides how
In the dataset, a value 1 in the outcome column "tested positive for DM" much from the previous memory cell should be neglected. The value
and a value 0, tested negative for DM. A brief description of the dataset close to 0 indicates most of the memory content of the previous time-
is shown in Tables 1 and 2. stamp will be forgotten at the current timestamp and the opposite
happens for the value close to 1. The operation of forget gate is defined
4. Models as (Rahman and Siddiqui, 2019):
ft = σg (wf x t + uf ht − 1 + bf ) (1)
The aim of the study is to develop a diabetes prediction model using
Convolutional LSTM that has not been applied yet in diabetes patient The input gate determines how much input from the current time-
classifications. Recently, different machine learning techniques like stamp will be added to the memory cell and is defined as (Rahman and
traditional LSTM, convolutional neural network (CNN) and their com- Siddiqui, 2019):
binations are widely used for diabetes patient classification based on it = σg (wi x t + ui ht − 1 + bi ) (2)
some clinically obtained parameters. These techniques achieve a pro-
mising accuracy in diabetes prediction although they face some chal- The control gate manages the process of updating memory cell
lenges like capturing the effect of the earliest input data and vanishing content from ct − 1 to ct by considering the output of forget and input gate

Table 1
Description of Pima Indian Diabetes dataset.
Sr. no. Selected Attributes from PIMA Indian dataset Description of selected attributes Range

1. Pregnancy Number of times a participant is pregnant 0–17

2. Glucose Plasma glucose concentration a 2 h in an oral glucose tolerance test 0–199
3. Blood pressure It consists of Diastolic blood pressure (when blood exerts into arteries between heart)(mm Hg) 0–122
4. Skin Thickness Triceps skinfold thickness (mm).It concluded by the collagen content 0–99
5. Insulin 2-Hour serum insulin (mu U/mL) 0–846
6. BMI Body mass index (weight in kg/(height in m)^2) 0–67.1
7. Diabetes pedigree Function An appealing attributed used in diabetes prognosis 0.078–2.42
8. Age Age of participants 21–81
9. Outcome Diabetes class variable, Yes represent the patient is diabetic and no represent patient is not diabetic 1/0

3
M. Rahman, et al.
Fig. 1. Diagram of Traditional LSTM.
that is defined as (Rahman and Siddiqui, 2019):
ct = ft × ct − 1 + it × σh (wc x t + uc ht − 1 + bc ) (3)
The final output of the current timestamp is produced by the output
gate, which is also responsible for updating cell state ht − 1 to ht (Rahman
and Siddiqui, 2019).
ot = σg (wo x t + uo ht − 1 + bo) (4)
ht = ot × σh (ct ) (5)
Here, σg and σh symbols denote sigmoid function and hyperbolic
tangent function respectively; w and u are weight values used to avoid
vanishing gradients problem. We have applied T-LSTM to classify dia-
betes patients using the Pima Indian Diabetes database. In our case, the
inputs are given to an input layer that is passed through two hidden
layers and a dense layer acts as an output layer that performs the final
classifications. Each layer of the model contains 100 of T-LSTM units
that captures the data pattern when the given inputs passed through
these units. An attention value for each input in each layer is estimated
that defines the importance of the given input in performing the final
prediction. With the help of an attention vector, the dense layer per-
forms the final prediction whether a patient has diabetes or not.
There has no connection between gates and the previous memory
content as shown in Fig. 1 that impedes from getting the impact of
previous memory content when the output gate is closed. This incon-
venience has adverse effects of classification and prediction perfor-
mance. The prime aim of this study is to apply Convolutional LSTM
(Conv-LSTM) for classifying diabetes patients and to point out how
Conv-LSTM overcomes the limitations faced by the T-LSTM including
CNN and CNN-LSTM.
4.2. Convolutional LSTM
The earliest Long Short-term Memory (LSTM) known as T-LSTM
cannot access the content of its previous memory cell when its output
gate is closed (Shi et al., 2015). We can solve this problem by adding an
extra connection between each gate and the previous memory content
that is known as a peephole connection. A variation of LSTM with
peephole connection is known as Convolutional LSTM (Conv-LSTM)
that allows all of the gates to utilize the previous memory cell content
even when the output gate is closed. The diagram of Conv-LSTM is
shown in Fig. 2 (Rahman and Siddiqui, 2019) that shows a similarity
with T-LSTM in terms of working principle. Here, each gate takes an
additional parameter (previous memory content ct-1) as input that
ensures the impact of earlier memory cells even when the output gate is
4
Computational Biology and Chemistry 88 (2020) 107329
closed. This connection guarantees the impact of the earliest input even
when the input sequence is long. The four gates of the Conv-LSTM
works as follows (Rahman and Siddiqui, 2019):
ft = σg (wf *x t + uf *ht − 1 + vf × ct − 1 + bf ) (6)
it = σg (wi *x t + ui *ht − 1 + vi × ct − 1 + bi ) (7)
ct = ft × ct − 1 + it × σh (wc *x t + uc *ht − 1 + bc ) (8)
ot = σg (wo*x t + uo*ht − 1 + vo × ct − 1 + bo) (9)
ht = ot × σh (ct ) (10)
In this study, we have developed a diabetes prediction model based
on Conv-LSTM that is applied over the Pima Indian Diabetes dataset.
4.3. Convolutional neural network (CNN)
A deep learning algorithm named as Convolutional neural network
(CNN) is similar to a neural network where some inputs are given to
each neuron. These neurons learn from data with the help of weight and
bias by performing some operations like dot product (Larabi-Marie-
Sainte et al., 2019). Initially, CNNs were used in image classification
but now it performs well in some other classification problems also.
To train and test a deep learning CNN model, the given inputs are
passed through a series of convolution layers with filters, pooling layer,
fully connected (FC) layers and finally apply Softmax function to make
the final prediction. The convolution layer extracts the underlying data
pattern by learning the relationship among the data. The pooling layer
reduces the number of parameters based on the importance of that
parameter. The fully connected layer produces the probability dis-
tribution over each class with the help of a Softmax function that de-
fines the final prediction result. The details of CNN-based model for
diabetes prediction is shown in Fig. 3.
The input data have been passed through a convolution layer that
captures the data pattern by learning different features. Then, a feature
map is generated by performing a dot product between given inputs and
weight values. Here, a bias value is added in each step to remove the
vanishing gradient problems. The convolution layer uses an activation
function named as ReLU (Rectified Linear Unit) that introduces non-
linearity in our convolutional network. Then the rectified feature maps
are given to the pooling layer that performs the down sampling. The
max pooling operation is applied here that takes the largest elements
from the rectified feature maps. The down sampled feature maps are
similarly passed through a convolution layer and pooling layer that
perform the same operation as the earlier convolution and pooling
M. Rahman, et al.
Fig. 2. Convolutional LSTM is derived from the T-LSTM by taking an extra connection from ct − 1 to each gate that is known as peephole connection.
layer. Then, the feature map is flattened into a vector and fed into a
fully connected layer like a neural network. The fully connected layer
combines all these vectors together to create a model and finally the
Softmax function classifies the patient into diabetes positive or not.
4.4. CNN-LSTM
The aforementioned CNN and LSTM both have strong predictive
power separately. Here, we have employed CNN in order to better
understand the hidden features of the dataset and combined with LSTM.
CNN-LSTM is the combination of CNN layers and LSTM layers that
ensures both advantages of CNN and LSTM. Fig. 4 shows the archi-
tecture of model based on CNN-LSTM for diabetes patients’ classifica-
tions. In our application, we feed inputs into a convolution layer that
extracts features from the given inputs and adds non-linearity in our
convolutional network with the help of an activation function ReLU.
Then, the generated rectified feature maps are passed through a pooling
layer that performs down sampling in order to decide the most acti-
vated features using max pooling operation. Then, the down sampled
feature maps are passed through a tile of LSTM layers that handle the
sequence processing. Lastly, a fully connected layer represents the
feature vector and uses it for classification, regression with the help of
Softmax function.
4.5. Boruta feature selection
Feature selection plays a very important role in deciding the
Fig. 3. Architecture of the developed CNN-based model for diabetes prediction.
5
Computational Biology and Chemistry 88 (2020) 107329
performance of a machine learning model. Very often, the modern
dataset contains too many features where most of them are irrelevant to
decide the outcome and their relevancy is not known in advance (Ali
et al., 2019). Presence of irrelevant features in a dataset has several
disadvantages- many machine learning algorithms display a fall of ac-
curacy, get slow down by taking too many resources and is simply in-
convenient to handle. Hence, selection of a small relevant feature set
from a large one is desirable for getting best possible classification re-
sult. In this study, we have applied Boruta feature selection algorithm
over the Pima Indian Diabetes database that returns glucose, BMI, in-
sulin, blood pressure, and age as important while pregnancy, diabetes
pedigree function, and skin thickness as unimportant features for clas-
sifying diabetes patients. Boruta is a feature selection algorithm that is
based on the foundation of random forest classifier. It adds randomness
to the system and collects results from the group of randomized samples
that assists to reduce the misleading impact of random fluctuations and
correlations. Therefore, the addition of extra randomness enables us to
divide all attributes into important and unimportant feature set. The
Boruta algorithm works as follows (Kursa and Rudnicki, 2010):
1 Add randomness to the provided dataset by creating shuffled copies
of all features that are called shadow features.
2 Run a random forest classifier on the extended dataset and measures
feature importance (Z score) to evaluate the importance of each
feature where higher means more important.
3 At every iteration, find the maximum Z score among the shadow
attributes and check whether a real feature has a higher Z score than
M. Rahman, et al.
Fig. 4. Architecture of the developed model based on CNN-LSTM for diabetes prediction.
the maximum Z score of its shadow features.
4 Feature with lower Z score than the maximum Z score of its shadow
features is deemed as unimportant and permanently remove them
from the information system.
5 Finally, the algorithm stops either when all features gets confirmed
or rejected or it reaches a specified limit of random forest runs.
Boruta is different from traditional feature selection algorithms as it
covers all features which are in some extent relevant to the outcome
while the traditional feature selection algorithms rely on a small subset
of features that generates a minimal error on a chosen classifier.
4.6. Hyperparameter optimization
Performance of a machine learning algorithm highly depends on the
optimum values of parameters. The optimum value of parameters are
obtained by tuning with different values and observer performance
where the algorithm returns better accuracy. Hyperparameter refers all
the parameters of a model that are not updated during the learning
phase but are used to mold the model to achieve the best accuracy.
Hyperparameter optimization also known as hyperparameter tuning is
the problem of selecting a set of optimal parameters that lowers the cost
function of the model. We have performed hyperparameter tuning over
the applied four models using grid search approach. Grid search also
called exhaustive search that looks through each combination of hy-
perparameters using permutation and combination. The performance of
grid search algorithm is measured using cross-validation on the training
set or evaluation on a held-out validation set. After performing all
possible combinations of hyperparameters, the grid search algorithm
returns the settings that achieved the highest accuracy in the validation
process along with the obtained accuracy. Hyperparameter optimiza-
tion of T-LSTM and Conv-LSTM has performed by keeping the value of
four parameters as follows:
[learning rate = 0.01, 0.03, 0.05], [batch size = 16, 32,64, 128],
[hidden units = 50, 100, 150], [epoch = 200, 250, 300, 350]
Besides, the hyperparameter tuning of CNN and CNN-LSTM has
performed over the following settings:
[learning rate = 0.01, 0.03, 0.05], [batch size = 16, 32,64, 128],
[hidden units = 50, 100, 150], [epoch = 200, 250, 300, 350], [kernel
size = 3*3, 5*5]
After performing hyperparameter optimization, grid search algo-
rithm returns optimal hyperparameters along with the highest accuracy
of each model. We have performed grid search algorithm over the
model by importing GridSearchCV model from sklearn package in py-
thon.
4.7. Evaluation
The performance of the applied models was evaluated by measuring
sensitivity (SN), specificity (SP), and accuracy (ACC). The sensitivity
6
Computational Biology and Chemistry 88 (2020) 107329
defines the ability of a model to accurately classify the true positive
samples while the specificity of a model is the capacity of identifying
the true negative samples. The accuracy of a model defines the ratio of
the number of patients correctly classified by the models. The formulas
are as follows (Zou et al., 2018):
TP
Sensitivity (SN ) =
TP + FN (11)
TN
Specificity (SP ) =
TN + FP (12)
TN + TP
Accuracy (ACC ) =
TN + TP + FP + FN (13)
Where true positive (TP) defines the number of classified positive pa-
tients those are actually positive. True negative (TP) represents the
number of predicted negative patients who are actually negative. False
positive (FP) is the number of classified positive patients who are ac-
tually negative. And false negative (FN) defines the number of identi-
fied negative patients who are actually positive. These parameters are
often estimated to assess the classification quality of models.
5. Experiment and architecture
Convolutional LSTM that has not been applied yet for classifying
diabetes patients. Hence, we have utilized this model to develop a novel
model for diabetes detection. The main concern before developing a
machine learning model is to enhance its performance. The perfor-
mance of a machine learning model highly depends on how much the
selected features are relevant to predict the outcome correctly.
Narrowing down the redundant features or inclusion of relevant fea-
tures has a positive impact on accuracy of a model (Ramaswami and
Bhaskaran, 2009). Besides, models face an overfitting problem when
too many features are used to train that lessens the capacity of gen-
eralizing patterns from new data (Karabulut et al., 2012). Additionally,
the computational complexity is also increased that requires more
memories and hardware. Hence, we have performed a feature selection
operation over the Pima Indian Diabetes dataset in order to extract the
most relevant features that have a very positive impact on deciding the
diabetes patients. We have applied the Boruta feature selection algo-
rithm, which is an improved version of Random Forest importance
measure algorithm (Degenhardt et al., 2019). Boruta feature selection
algorithm takes multi-variable relationships into consideration for de-
ciding all relevant features and it performs well for both regression and
classifications problems (Fortino et al., 2014). Here, the applied Boruta
algorithm returns Glucose, BMI, Blood Pressure, Insulin and Age as
relevant features for building an accurate and robust model in order to
classify diabetes patients. Before performing the Boruta feature selec-
tion algorithm, we have cleaned data since some features like glucose,
blood pressure, skin thickness, insulin, and BMI in our dataset have
some zero (0) values that are dealt with the median value of that
M. Rahman, et al. Computational Biology and Chemistry 88 (2020) 107329

Fig. 5. Architecture of the developed model based on Conv-LSTM for diabetes prediction.

attribute. The architecture of our model is shown in Fig. 5. 6. Results and discussion
We have incorporated three layers of Convolutional LSTM (Conv-
LSTM) to capture the underlying features of data and a fully connected The study was aimed at applying the Traditional LSTM (T-LSTM),
dense layer that represents the captured hidden data patterns. Initially, Convolutional Neural Network (CNN), CNN-LSTM, and Convolutional
the preprocessed data are passed through an Embedding layer that LSTM (Conv-LSTM) separately for building diabetes patients classifi-
transforms each given input into a distributed representation of length cation models. The performance of these four models were observed
100, which is then fed through three Con-LSTM layers. Each layer of over the Pima Indian Diabetes dataset. We have performed hy-
Conv-LSTM contains hundred hidden units and processes the received perparameter optimization using grid search algorithm to choose the
input in both directions in order to capture the underlying data features optimal hyperparameters that achieved the highest accuracy. The re-
effectively. The first layer of three Conv-LSTM layers acts as an input sults of grid search over the four models is shown in Tables 1–3. From
layer while the rest two layers are hidden layers. While the data are all possible combinations, the top 3 combinations that have achieved
passed through these three layers, a conditional probability known as the highest test score are shown in table.
attention is calculated for each input that determines the importance of Tables 1 and 2 shows that the both Conv-LSTM and T-LSTM have
that input in performing patient classification. The estimated attention obtained the highest test score of 98.24 and 91.31 percent respectively
for each input is stored in an attention vector that is used in the fully with the optimal hyperparameters like learning rate 0.01, batch size 32,
connected dense layer for doing accurate classification. Then, the cap- hidden units 100 and epoch 300. According to Table 3, the CNN-LSTM
tured feature vector is fed through the fully connected dense layer that and CNN models have achieved highest accuracy of 94.87 and 89.02
represents the data pattern and uses that pattern for final classification percent respectively with the same configurations of above two models
of diabetes patients. Here, a Softmax function is used that generates a by keeping kernel size 5*5. Hence, we have considered learning rate
probability distribution and this probability distribution defines whe- 0.01, batch size 32, hidden units 100, epoch 300 and kernel size 5*5 for
ther a patient is diabetes positive or not. each respective model to perform the final classification. Here, the
Initially, we have applied our model over the Pima Indian Diabetes learning rate defines how much of weight is adjusted in the model with
dataset by splitting the dataset into separate training and testing da- respect to the loss gradient; batch size is the number of sample inputs
taset. Later, we have observed the performance of our model by using passed through the model at a time; epoch defines how many times a
the five-fold cross validation technique over the same dataset. Our one machine learning model traverses through the same dataset.
objective is to find out the optimal parameter configuration where the We have applied these models over the mentioned dataset in two
model returns highest accuracy. Therefore, We have splitted the dataset separate ways such as keeping the training-testing set separately and
on three different points for building training and testing dataset such using five-fold cross validation technique to evaluate the performance.
as 80 % train and 20 % test, 75 % train and 25 % test, 70 % train and 30 Experimental results of both cases are shown in Table 4 that shows our
% test. Three different batch sizes such as 16, 32 and 64 are considered Conv-LSTM-based model has performed better in both cases compared
while three different learning rates like 0.01, 0.03 and 0.05 are taken to other three models. When the dataset has been splitted at point 0.2
into account for tuning the model. We have performed all experiments for training and testing data, the Conv-LSTM model has achieved the
for epoch 200, 250 and 300 separately to figure out the impact of the highest accuracy of 91.38 % that is greater than the other three models.
number of epochs on the model's performance.
Here, the size of the dataset is not so large. Therefore, there is a
chance of occurring overfitting problem that need to be dealt with. We Table 3
Hyperparameter optimization of CNN-LSTM and CNN.
have tuned the model by adding more layers and found that when the
hidden layer is more than two, the overfitting problem occurs. Hence, CNN-LSTM
we have kept two hidden layers in the model. Additionally, we have
Learning rate Mini Batch Hidden Epoch Kernel size Mean test
added dropout layers that randomly sets output features of a layer to
size units score
zero. The addition of dropout layers have exhibited that the model with
dropout layers starts overfitting at later than the model without dropout 0.01 32 100 300 5*5 94.87
layers. We have implemented all four models such as T-LSTM, CNN, 0.01 32 100 300 3*3 92.74
0.01 64 100 300 5*5 91.28
CNN-LSTM, and Conv-LSTM using a Python library called Keras which
CNN
is backed by TensorFlow. 0.01 32 100 300 5*5 89.02
0.01 64 100 300 3*3 87.19
0.03 32 100 300 3*3 85.98

7
M. Rahman, et al. Computational Biology and Chemistry 88 (2020) 107329

Table 4 capturing long data-dependency that is quite hard using the other
Diabetes prediction accuracy of different models with selected features. aforementioned models when the dataset is large.
Evaluation Models Accuracy Sensitivity (SN) Specificity (SP) We have performed several experiments to outline the combined
Techniques (ACC) effects of these parameters over classification accuracy that are shown
in Fig. 6. Figure shows that batch size and learning rate (Fig. 6(a)),
Separate T-LSTM 85.69 84.75 85.70
batch size and splitting point (Fig. 6(b)), batch size and epoch number
Training- CNN 82.14 83.04 82.34
Testing Set CNN-LSTM 88.47 87.78 89.47
(Fig. 6(c)) have significant combined impacts on the performance of the
Conv-LSTM 91.38 92.19 92.41 model. The combined effect of splitting point and epoch number
Five-fold Cross- T-LSTM 90.89 89.69 91.78 (Fig. 6(d)), learning rate and epoch number (Fig. 6(e)), learning rate
Validation CNN 88.73 86.79 88.07 and splitting point (Fig. 6(f)) are less important in regards to the
CNN-LSTM 92.06 93.17 92.11
model's accuracy.
Conv-LSTM 97.26 97.28 97.09
In addition, the performance of these four applied models has been
reported on the basis of different epoch numbers from 100 to 400. Fig. 7
Table 5 shows that the four models have achieved better accuracy at whatever
Diabetes prediction accuracy of different models with all features. the epoch number is when the five-fold cross validation is used instead
of keeping training and testing data separately. Fig. 7 also dictates that
Evaluation Models Accuracy Sensitivity (SN) Specificity (SP)
Techniques (ACC)
each of four models has provided the highest accuracy when the model
has been trained for about 300 epochs. The reason behind the less ac-
Separate T-LSTM 83.65 83.69 84.07 curacy of these models is underfitting and overfitting problems oc-
Training- CNN 81.47 81.25 80.98 curred when the models are trained for less than or greater than 300
Testing Set CNN-LSTM 84.31 84.26 85.04
Conv-LSTM 86.61 85.96 86.25
epochs respectively. When the overfitting problem occurs, the model
Five-fold Cross- T-LSTM 91.69 90.86 91.09 tries to memorize the data and cannot generalize the new data effi-
Validation CNN 89.69 90.08 90.41 ciently. The model performs poorly over the training and new data
CNN-LSTM 91.65 91.02 93.69 when the underfitting problem occurs.
Conv-LSTM 94.23 93.98 94.04
However, we could make a conclusion after all those discussions
that the Conv-LSTM model has classified the diabetes patients more
accurately than the other three models. The reason behind is the ar-
The score of Sensitivity and Specificity at Table 4 indicates the accuracy
chitecture of Conv-LSTM that enables the model to capture the long
of predicting the positive and negative patients of Conv-LSTM is 92.19
term data dependency with the help of an extra connection called
and 92.41 percent respectively that is higher in comparison with other
peephole, which is absent in the traditional LSTM. The combination of
three models. Besides, the results also indicate that the model based on
CNN-LSTM performs better than the individual CNN and LSTM because
Conv-LSTM has classified diabetes patients more accurately when five-
it integrates the power of both CNN and LSTM to make a final decision.
fold cross validation is used instead of keeping the training and testing
In the combination of CNN-LSTM, the CNN helps to extract the features
set separately.
and LSTM assists to remember the long data pattern efficiently that
We have also applied these model to classify diabetes patients by
make the model more capable to generalize the new data.
using all eight features such as Pregnancies, Glucose, Blood Pressure,
BMI, Skin Thickness, Insulin, Diabetes Pedigree Function, and Age of
the Pima Indian Diabetes dataset in order to find the impact of Boruta
7. Conclusions
feature selection algorithm over the performance of our model. The
results of these experiments is shown in Table 5 that dictates that all
Diabetes is caused by the unbalanced insulin secretion or its im-
these models classify diabetes patients more accurately when the sig-
paired biological impacts or both that becomes visible when the glucose
nificant features are used instead of all eight features. The highest ac-
level in blood is higher than the normal level. When the development of
curacy is 94.23 % obtained by the Conv-LSTM when all features and
a reliable machine learning model for diabetes detection is necessary,
five-fold cross validation technique are used, which is less than the
we have applied four different models such as Conv-LSTM, CNN-LSTM,
accuracy obtained by the model with significant features are used.
T-LSTM, and CNN for classifying diabetes patients over the Pima Indian
We have compared the performance of our developed Convolutional
Diabetes dataset. The novelty of this work is that the Convolutional
LSTM-based model with that of the state-of-the-art models as shown in
LSTM (Conv-LSTM) is applied for the first time in this regard as it
Table 6. Here, all referred models in Table 6 were applied over the Pima
performs well in various classification and prediction problems. We
Indian Diabetes dataset. The comparison dictates that our convolutional
have applied these models over the mentioned dataset in two different
LSTM-based model has classified diabetes patients more accurately
ways-by keeping the training and testing data separately and using the
while the decision tree and the combination of SVM and NN have
five-fold cross validation technique to evaluate the models. We have
classified with a less accuracy in compared to our model. The reason
utilized the Boruta algorithm, which is based on the random forest to
behind the higher performance of Conv-LSTM model is the ability of
select the most significant features among the eight features of the

Table 6
Performance comparison of our model with state-of-the-art models.
Model Name Accuracy (%) Reference

Deep Learning Architecture 88.41 (Ashiquzzaman et al., 2017)

ANN 90.34 (Naz and Ahuja, 2020)
Decision Tree 96.62 (Naz and Ahuja, 2020)
LSTM 86 (Massaro et al., 2019)
Recurrent Deep Neural Network (RNN) 81 (Ashiquzzaman et al., 2017)
Support Vector Machine (SVM) and Neural Network (NN) 96.09 (Gill and Mittal, 2016)
Random Forest 94 (Yuvaraj and SriPreethaa, 2019)
Naïve Bayes 91 (Yuvaraj and SriPreethaa, 2019)
Convolutional LSTM 97.26 Our Study

8
M. Rahman, et al. Computational Biology and Chemistry 88 (2020) 107329

Fig. 6. Combined effects of batch size, epoch number, learning rate, and splitting point over the accuracy.

Fig. 7. Performance observation of the applied models over different epoch numbers.

9
M. Rahman, et al.
database. We have performed hyperparameter optimization to find the
optimal values of parameters. After experimental analysis, we have
found that each of four models performs better when the five-fold cross
validation technique is used instead of splitting the dataset for training
and testing. We have also reported that the Conv-LSTM model outper-
forms the other three models in both cases in terms of classification
accuracy. Comparison of the developed model with the state-of-the-art
models shows the superiority of our model.
Declaration of Competing Interest
The authors declares that there is no conflict of interest to any
person or any organization.
References
Agatonovic-Kustrin, S., Beresford, R., 2000. Basic concepts of artificial neural network
(ANN) modeling and its application in pharmaceutical research. J. Pharm. Biomed.
Anal. 22 (5), 717–727 Elsevier 01-Jun-.
Ali, Z., Hussain, I., Faisal, M., Elashkar, E.E., Gani, S., Shehzad, M.A., 2019. Selection of
appropriate time scale with Boruta algorithm for regional drought monitoring using
multi-scaler drought index. Tellus A Dyn. Meteorol. Oceanogr. 71 (1 January),
1604057.
Allam, F., Nossai, Z., Gomma, H., Ibrahim, I., Abdelsalam, M., 2011. A recurrent neural
network approach for predicting glucose concentration in type-1 diabetic patients.
IFIP Advances in Information and Communication Technology, vol. 363. pp. 254–259
AICT, no. PART 1.
Apoorva, S., Aditya, K., Snigdha, S.P., Darshini, P., Sanjay, H.A., 2020. Prediction of
Diabetes Mellitus Type-2 Using Machine Learning. pp. 364–370.
Ashiquzzaman, A., et al., 2017. Reduction of overfitting in diabetes prediction using deep
learning neural network. Lecture Notes in Electrical Engineering, vol. 449. pp. 35–43.
Clark, N.G., Fox, K.M., Grandy, S., 2007. Symptoms of diabetes and their association with
the risk and presence of diabetes: findings from the study to help improve early
evaluation and management of risk factors leading to diabetes (SHIELD). Diabetes
Care 30 (11 November), 2868–2873.
Degenhardt, F., Seifert, S., Szymczak, S., 2019. Evaluation of variable selection methods
for random forests and omics data sets. Brief. Bioinform. 20 (2), 492–503.
Fortino, V., Kinaret, P., Fyhrquist, N., Alenius, H., Greco, D., 2014. A robust and accurate
method for feature selection and prioritization from multi-class OMICs data. PLoS
One 9 (9 September), e107801.
Gill, N.S., Mittal, P., 2016. A Computational Hybrid Model With Two Level Classification
Using SVM and Neural Network for Predicting the Diabetes Disease.
Guo, D., Zhang, D., Li, N., Zhang, L., Yang, J., 2010. Diabetes identification and classi-
fication by means of a breath analysis system. Lecture Notes in Computer Science
(Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in
Bioinformatics), vol. 6165. LNCS, pp. 52–63.
Kamble, A.K., Manza, R.R., Rajput, Y.M., 2016. Review on Diagnosis of Diabetes in Pima
Indians. undefined. .
Karabulut, E.M., Özel, S.A., İbrikçi, T., 2012. A comparative study on the effect of feature
selection on classification accuracy. Procedia Technol. 1 (January), 323–327.
10
Computational Biology and Chemistry 88 (2020) 107329
Kotsiantis, S.B., 2013. Decision trees: a recent overview. Artif. Intell. Rev. 39 (4), 261–283
Springer,29-Apr-.
Kursa, M.B., Rudnicki, W.R., 2010. Feature selection with the boruta package. J. Stat.
Softw. 36 (11 September), 1–13.
Larabi-Marie-Sainte, Aburahmah, Almohaini, Saba, 2019. Current techniques for diabetes
prediction: review and case study. Appl. Sci. Basel (Basel) 9 (21 October), 4604.
Li, Y., Li, H., Yao, H., 2018. Analysis and study of diabetes follow-up data using a data-
mining-Based approach in new urban area of Urumqi, Xinjiang, China, 2016-2017.
Comput. Math. Methods Med. 2018.
Massaro, A., Maritati, V., Giannone, D., Convertini, D., Galiano, A., 2019. LSTM DSS
automatism and dataset optimization for diabetes prediction. Appl. Sci. Basel (Basel)
9 (17 August), 3532.
Mhaskar, H.N., Pereverzyev, S.V., van der Walt, M.D., 2017. A deep learning approach to
diabetic blood glucose prediction. Front. Appl. Math. Stat. 3 (July).
Muzaffar, S., Afshari, A., 2019. Short-term load forecasts using LSTM networks. Energy
Procedia 158, 2922–2927.
Naz, H., Ahuja, S., 2020. Deep learning approach for diabetes prediction using PIMA
Indian dataset. J. Diabetes Metab. Disord. (April), 1–13.
Rahman, S.A., Adjeroh, D.A., 2019. Deep learning using convolutional LSTM estimates
biological age from physical activity. Sci. Rep. 9 (1 December), 1–15.
Rahman and Siddiqui, 2019. An optimized abstractive text summarization model using
peephole convolutional LSTM. Symmetry (Basel). 11 (10 October), 1290.
Ramachandran, A., 2014. Know the signs and symptoms of diabetes. Indian J. Med. Res.
140 (5), 579–581 Indian Council of Medical Research 01-Nov-.
Ramaswami, M., Bhaskaran, R., 2009. A Study on Feature Selection Techniques in
Educational Data Mining. Dec. .
Shi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W., Woo, W., 2015. Convolutional LSTM
network: a machine learning approach for precipitation nowcasting. Adv. Neural Inf.
Process. Syst. 2015 (January), 802–810 Jun.
Singh, P.P., Prasad, S., Das, B., Poddar, U., Choudhury, D.R., 2018. Classification of
diabetic patient data using machine learning techniques. Advances in Intelligent
Systems and Computing, vol. 696. pp. 427–436.
Sneha, N., Gangil, T., 2019. Analysis of diabetes mellitus for early prediction using op-
timal features selection. J. Big Data 6 (1 December).
Staudemeyer, R.C., Morris, E.R., 2019. Understanding LSTM – a Tutorial Into Long Short-
term Memory Recurrent Neural Networks. Sep. .
Sun, Q., Jankovic, M.V., Bally, L., Mougiakakou, S.G., 2018. Predicting blood glucose
with an LSTM and Bi-LSTM based deep neural network. In: 2018 14th Symposium on
Neural Networks and Applications. NEUREL 2018.
Swapna, G., Vinayakumar, R., Soman, K.P., 2018. Diabetes detection using deep learning
algorithms. Ict Express 4 (4 December), 243–246.
Yamashita, R., Nishio, M., Do, R.K.G., Togashi, K., 2018. Convolutional neural networks:
an overview and application in radiology. Insights Imaging 9 (4), 611–629 Springer
Verlag 01-Aug-.
Yuvaraj, N., SriPreethaa, K.R., 2019. Diabetes prediction in healthcare systems using
machine learning algorithms on Hadoop cluster. Cluster Comput. 22 (1
January), 1–9.
Zhang, Y., 2012. Support vector machine classification algorithm and its application.
Communications in Computer and Information Science, vol. 308. CCIS, pp. 179–186
no. PART 2.
Zhu, G., et al., 2019. Redundancy and attention in convolutional LSTM for gesture re-
cognition. IEEE Trans. neural networks Learn. Syst.(June).
Zou, Q., Qu, K., Luo, Y., Yin, D., Ju, Y., Tang, H., 2018. Predicting diabetes mellitus with
machine learning techniques. Front. Genet. 9 (November).