In Tell I Health

IntelliHealth: An intelligent medical decision support
system using a novel multi-layer classifier ensemble

framework
1
Overview
Introduction
Problem Statement
Existing Approaches
Research Methodology
Experimental Results
IntelliHealth Application
Case Studies
Conclusion
Future Work
2
Motivation
• High accuracy is of prime importance in medical field for

disease diagnosis
• Reduce the margin of diagnostic errors
• Solution: Medical data mining
3
Introduction
4
Why Medical Data Mining
• In 2012, worldwide digital healthcare data was estimated to be equal to
500 petabytes and is expected to reach 25,000 petabytes in 2020 (1024
Gigabytes = 1 Terabyte · 1024 Terabytes = 1 Petabyte)
• Learn from the past to become better in the future
Sun, J., & Reddy, C. K. (2013, August). Big data analytics for healthcare. In Proceedings of the 19th ACM SIGKDD international
conference on Knowledge discovery and data mining (pp. 1525-1525). ACM.
5
Why Medical Data Mining
• Manually impossible to process such large amount of data
• Identify hidden patterns or structures of historical data
• Assist doctors in medical decision making
Sun, J., & Reddy, C. K. (2013, August). Big data analytics for healthcare. In Proceedings of the 19th ACM SIGKDD international
conference on Knowledge discovery and data mining (pp. 1525-1525). ACM.
6
Medical Data Mining
Analytics
Integrate Repository Knowledge
Pattern
Base
Extraction
Clinical
Decision
Support
System
Hunink, H. and Glasziou, P. (2009) Decision Making in Health and Medicine: Integrating Evidence and Values (7th ed.). Cambridge
England: Cambridge
7
Clinical Decision Support System
 A Computer based clinical decision support system
(CDSS) provides
• Clinical knowledge and patient related information
• Intelligently filtered and presented at appropriate time
• Enhance patient’s care
Osheroff, JA, Pifer, EA, Sitting, DF, Jenders, RA, Teich, JM. ‘Clinical decision support implementers’ workbook”. Chicago:
HIMSS, 2004
8
An Overview of Machine Learning
Techniques for CDSS
Naive Bayes
Classification
techniques
Decision trees
Support vector
machine
Machine Clustering
learning techniques
K nearest
neighbor
K mean
clustering
Majority voting
SOM
Ensemble Bagging
techniques Expectation
Max.
AdaBoost
Foster, K. R., Koprowski, R., & Skufca, J. D. (2014). Machine learning, medical diagnosis, and biomedical engineering research-
commentary.Biomedical engineering online, 13(1), 94.
9
Machine Learning Techniques for CDSS
 Classification Techniques
 New objects are labeled based on the training set
 Examples:
 Naïve Bayes,
 SVM,
 Decision trees etc
Manning, E. M., Holland, B. R., Ellingsen, S. P., Breen, S. L., Chen, X., & Humphries, M. (2016). Comparison of three Statistical
Classification Techniques for Maser Identification. arXiv preprint arXiv:1603.06395.
10
 Clustering Techniques
 Identify hidden structure from unlabeled data
 Examples:
• k-means,
• SOM (Self Organizing Map),
• AGNES etc
Machnik, Ł. (2015). Documents Clustering Techniques. Annales UMCS Sectio AI Informatica, 2(1), 401-411.
11
 Ensemble Techniques
• Aggregate predictions made by multiple classifiers
• Predict class label of previously unseen records
 Examples include:
• Majority voting
• Weighted voting
• Bagging etc
Sarumathi, S., Shanthi, N., & Ranjetha, P. (2016). Analysis of Diverse Cluster Ensemble Techniques. World Academy of Science,
Engineering and Technology, International Journal of Computer, Electrical, Automation, Control and Information Engineering, 9(11),
2208-2218.
12
Why Ensemble
• Each individual or single classifier has some limitations
• Trade-offs between issues such as accuracy, training

time, robustness, and scalability etc
• “No Free Lunch” Theorem

• No single algorithm wins all the time!
• There is a need of an ensemble model that can combine

independent, diversified models to improves accuracy
13
(Literature Review)
Existing Approaches
14
Existing Approaches
• Several machine learning techniques are used for
disease diagnosis
• Some of these state of the art techniques are given here
• Open data standard is the norm of Medical Sciences,

• Consider pervious work only on publically available benchmark
datasets
15
Examples of Benchmark Datasets
No. Datasets Dataset name Reference
1 Heart disease Cleveland heart disease http://archive.ics.uci.edu/ml/datase

ts.html
SPECT
http://www.eric.univ-
SPECTF lyon2.fr/~ricco/dataset/heart_disea
se_male
Statlog
Eric
https://www.lri.fr/~antoine/Courses
2 Breast cancer UMC breast cancer /Master-ISI/TD-TP/breast-
cancer.arff
Wisconsin breast cancer
Wisconsin Diagnostic breast
cancer
Wisconsin Prognostic breast
cancer
http://biostat.mc.vanderbilt.edu/wik
3 Diabetes Pima Indian Diabetes Dataset
i/pub/Main/DataSets/diabetes.html
Biostat Diabetes Dataset
https://archive.ics.uci.edu/ml/datas
4 Liver Bupa Liver disease dataset ets/ILPD+(Indian+Liver+Patient+D
ataset)
Indian Liver patient dataset
https://archive.ics.uci.edu/ml/datas
5 Hepatitis Hepatitis disease dataset ets/Hepatitis
16
Machine Learning Techniques Applied on
Benchmark Heart Disease Datasets
Author/Reference Year Technique Accuracy
K-Mean based on Maximal Frequent Itemset

74%
Algo (MAFIA)
Thenmozhi, K. et al., [1] 2015
K-Mean based on MAFIA with ID3 85%
Bashir,S., et al., [2] 2014 Majority Voting Ensemble 82%
Cascaded Neural Network 85%

Chitra, R., et al., [3] 2013
Support Vector Machine 82%
Gain ratio Decision Tree 79.1%
Shouman, M., et al., [4] 2013 Naïve Bayes 83.5%
K Nearest Neighbor 83.2%
Support Vector Machine Ensemble 85.05%

Ghumbre, S., et al., [5] 2011
Radial Basis Function 82.24%
Shouman, M., et al., [6] 2011 Bagging with Gain Ratio Decision Trees 84.1%
Ozcift, A., [7] 2011 Random Forest 89%
17
Benchmark Breast Cancer Datasets
Chaurasia, V. et al. [8] 2015 Sequential Minimal Optimization 96.2%
Support Vector Machine Ensemble 75.00%

K, A.A., et. al. [9] 2013
Random forest 74.00%
SMO+J48+NB+KNN (Majority voting) 96.28%
Salama, G.I., et al. [10] 2012 SMO+J48+MLP+KNN (Majority voting) 97%
SMO 77.31%
Lavanya, D., et al. [11] 2012 Decision Tree Ensemble 95.96%
Luo, S., er al. [12] 2012 MultiBoosting Decision Tree 87.9%
S. Aruna, et al. [13] 2011 SVM-RBF Kernel 96.84%
Christobel, A., et al. [14] 2011 Support Vector Machine Ensemble 96.84%
Lavanya, D., et al. [15] 2011 CART with Feature Selection 94.56%
18
Benchmark Diabetes Datasets
Han, L., et al. [16] 2015 Support Vector Machine 91.2%
Gandhi, K.K. et al. [17] 2014 F-score Feature Selection+ SVM 75%
Sliding Window Bayesian Model

Stahl, F. et al. [18] 2014 71%
Averaging
Genetic programming with

Aslam, M.W. et al. [19] 2013 79%
comparative partner selection
NirmalaDevi, M. et al. [20] 2013 K Nearest Neighbor 92.4%
K Nearest Neighbor with Imputation

Christobel. Y. A. et al. [21] 2012 73.38%
and Scaling
Zolfaghari, R. [22] 2012 BP Neural Network and SVM 88%
Lee, C. [23] 2011 Fuzzy Diabetes Ontology 75%
Ozcift, A. et al. [24] 2011 Random Forest 74.47%
Verma, B. et al. [25] 2011 Neural Network Ensemble 80%
19
Benchmark Liver Disease Datasets
Vijayarani, S. et al. [26] 2015 Support Vector Machine 61.2%
Naïve Bayes 70.6%
Seker, S.E. et al. [27] 2014 Bagging 72%
Pahareeya, J. et al. [28] 2014 Random forest 72.4%
Jin, H. et al. [29] 2014 Decision tree 69.4%
Sugawara, K. et al. [30] 2013 Self-Organizing Map 71%

Kumar, Y. et al. [31] 2013 Rule based classification model 72%
Ramana, B.V. et al. [32] 2012 Neural Network 73%
SVM ensemble 72.5%

Ramana, B.V. et al. [33] 2011 Back propagation NN 66.66%
Support Vector Machine 62.6%
Karthik, S. et al. [34] 2011 Naïve Bayes 55%
Multilayer Perceptron 70.5%
RBF network 61.5%
20
Benchmark Hepatitis Dataset
Krawczyk, B. et al. [35] 2014 SVM ensemble 85%
Pushpalatha, S. et al. [36] 2014

Naïve Bayes 80%
El Houby, E.M.F. [37] 2013

Associative Classification 86%
Karthikeyan, T. et al. [38] 2013

Random Forest 83%
Kaya, Y. et al. [39] 2013

Rough set-Extreme Learning Machine 86.4%
Kumar.M, V. et al. [40] 2012

Lin, S. et al. [41] 2011

Majority Voting Ensemble 82%
Eldin, A.M.S.A.G. [42] 2011

Decision Tree 86%
Javad, S.S. [43] 2011

SVM + Simulated Annealing 86.25%
Oh, S., et al. [44] 2011

Bagging Ensemble 85%
21
Gaps Identified in Existing Approaches
• Most of the work is done on single classifiers
• Different algorithms perform differently depending on

data. For example:
• Naïve Bayes needs to categorize continuous variables
• SVM needs to transform nominal data into numerical
• Linear regression limits the prediction of numeric ouput
• Neural network is suitable for discrete and continuous data etc
• None of them appears to be globally superior over the

others
22
Gaps Identified in Existing Approaches
• No single framework for diagnosis of multiple diseases with
high accuracy
• There is no agreement on which methodology is better than

others
• One classifier performs well in one dataset/disease, another

classifier can outperforms for different dataset/disease
• The proposed approach exploits the potentials of several

classifiers in an ensemble model
23
Ensemble Model
• Aggregate predictions made by multiple classifiers
• Two important Factors

1. Selection of classifiers
2. Combination of classifiers
• The main purpose of an ensemble model is

Maximizing accuracy and diversity
Rokach, L. (2005). Ensemble methods for classifiers. In Data Mining and Knowledge Discovery Handbook (pp. 957-980). Springer
US.
24
Selection of Classifiers
• The individual classifiers composing an ensemble must
be accurate and diverse:
• Accurate classifier
• Accurate classifier has an error rate better than the
random when guessing new examples
• Diverse classifiers
• Two classifiers are diverse if they use different
training data or different learning algorithms and
produce predictions independently
Rokach, L. (2005). Ensemble methods for classifiers. In Data Mining and Knowledge Discovery Handbook (pp. 957-980). Springer
US.
25
Combination of Classifiers
• Different ways to combine output of single classifiers for
maximum accuracy
• Examples
• Majority Voting
• Bagging
• AdaBoost
• Stacking etc
Hou, J., Xu, E., Xia, Q., & Qi, N. M. (2015). Evaluating classifier combination in disease classification. Pattern Analysis and
Applications,18(4), 799-816.
26
Research Methodology
27
Proposed Approach
• Different variations of ensemble models have been
proposed
• Each ensemble model results in high disease diagnosis

accuracy than the last one
28
Medical Data Preprocessing
29
Why Data Preprocessing
• Patient records consist of clinical, lab parameters, results
of particular investigations, specific to disease
• Incomplete: lacking attribute values
• Noisy: containing errors or outliers
• Irrelevant data: containing irrelevant attributes

specific to disease
Hu, Y. H., Lin, W. C., Tsai, C. F., Ke, S. W., & Chen, C. W. (2015). An efficient data preprocessing approach for large scale medical
data mining. Technology and Health Care, 23(2), 153-160.
30
• Missing Value Replacement
• KNNimpute method
• Find the instances most similar to the test instance
• Euclidean distance based similarity
k
d(x,y)= 
i 1
( xi  y i ) 2
• Missing attribute->continuous->replace with mean value of k

nearest neighbors
• Missing attribute->categorical->majority vote of k nearest
neighbors
García-Laencina, P. J., Abreu, P. H., Abreu, M. H., & Afonoso, N. (2015). Missing data imputation on the 5-year survival prediction of
breast cancer patients with unknown discrete values. Computers in biology and medicine,59, 125-133.
31
• Outlier Detection and Elimination
• Euclidean Distance based outliers
k
d(x,y)=  (x  y )
i 1
i i
2
• Distance of a point from its k nearest neighbors

• Ranking on the basis of its distance to k nearest neighbors
• A point is an outlier, if its kNN distance is much larger then that
of other data points
Aggarwal, C. C. (2015). Outlier analysis. In Data Mining (pp. 237-263). Springer International Publishing.
32
• Feature Selection
• Greedy forward feature selection [*]
• Select subset of features based on performance
gain
*Ververidis, D., & Kotropoulos, C. (2011). Fast and accurate sequential floating forward feature
selection with the Bayes classifier applied to speech emotion recognition. Signal
Processing, 88(12), 2956-2970.
33
All our proposed models will use these techniques

for preprocessing of medical data
34
Proposed Ensemble Model 1
MV5
35
Proposed Ensemble Model 1 - MV5
• Majority Voting Ensemble based on Five Classifiers
(MV5)
• For construction of proposed ensemble model, following

steps are followed:
1. Selection of classifiers
2. Combination of classifiers
36
• Initially a single disease was considered i.e. Heart

Disease
• Performed literature survey of almost 100 research

papers published in last five years related to heart
disease diagnosis
• Top five classification techniques are selected based on

accuracy and diversity
37
• From literature, following five heterogeneous
classifiers are selected:
• Naïve Bayes (NB)
• Decision Tree Induction using Information Gain (DT-IG)
• Decision Tree Induction using Gini Index (DT-GI)
• Support Vector Machine (SVM)
• K Nearest Neighbor (KNN)
• Initially started with basic and simple voting ensemble
38
• Heterogeneous classifiers, complement each other
• Naïve Bayes
P(X | C )P(C )
P(C | X)  i i
i P(X)
x=(x1….xn) represents the features and i is possible classes
• Class conditional independence, therefore loss of accuracy

• Practically, dependencies exist among medical data ->Resolved by
KNN
• Efficient in terms of CPU and memory consumption
• Requires very small amount of training data and time
• Need to categorize continuous predictors -> Resolved by Decision tree
Parthiban, G., Rajesh, A., & Srivatsa, S. K. (2011). Diagnosis of heart disease for diabetic patients using naive bayes
method. International Journal of Computer Applications, 24(3), 7-11.
39
• Decision Trees
• Powerful and popular tools for classification and prediction
• Faster learning, high accuracy
• May suffer from overfitting -> Resolved by SVM
• Handle continuous and categorical variables -> Resolve
Naïve Bayes issue
Nahar, J., Imam, T., Tickle, K. S., & Chen, Y. P. P. (2013). Computational intelligence for heart disease diagnosis: A medical
knowledge driven approach. Expert Systems with Applications, 40(1), 96-104.
40
• Decision Tree using Information Gain
m
Entropy: Info( D)    p log
i 1
i 2 ( pi )
pi is class probability,
m is number of classes
v
| Dj |
Entropy of each attribute: InfoA ( D)   | D |  I (D )
j 1
j split D into v
partitions
Information Gain: Gain(A)  Info(D)  InfoA(D)
Select attribute with highest information gain
41
• Decision Tree using Gini Index
n
Gini index gini( D)  1   p 2j n is no. of classes
j 1
|D1| |D |
Gini of each attribute giniA ( D)  gini( D1)  2 gini( D2)
|D| |D|
Total Gini gini( A)  gini(D)  giniA(D)
Select attribute with lowest Gini index
42
• K Nearest Neighbor
k
Euclidean distance d (x,y)=
 (x  y )
i 1
i i
2
• Determine similarity between attributes -> Neighbors determine the

class label for unknown instance -> Resolves Naïve Bayes issue
• Very accurate
• Accuracy degrades with increase of irrelevant attributes -> Feature
selection
• Simple learning
• Robust to noise
Gagliardi, F (2011). "Instance-based classifiers applied to medical databases: Diagnosis and knowledge extraction". Artificial
Intelligence in Medicine 52 (3): 123–139
43
• Support Vector Machine
• SVM attempts to find maximum margin to separate classes
Sartakhti, J. S., Zangooei, M. H., & Mozafari, K. (2012). Hepatitis disease diagnosis using a novel hybrid method based on support
vector machine and simulated annealing (SVM-SA). Computer methods and programs in biomedicine, 108(2), 570-579.
44
• High classification and prediction accuracy
• Alternative to neural network
• Handles both continuous and categorical variables ->Resolves Naïve
Bayes issue
• Decreases the overfitting issue -> Resolves decision tree issue
• Sensitive to noise -> Preprocessing
In any scenario where one classifier has some limitation, the

other classifier performs well, consequently giving better
performance
Sartakhti, J. S., Zangooei, M. H., & Mozafari, K. (2012). Hepatitis disease diagnosis using a novel hybrid method based on support
vector machine and simulated annealing (SVM-SA). Computer methods and programs in biomedicine, 108(2), 570-579.
45
Majority Voting Ensemble
The majority voting ensemble will output the class which has highest
number of votes. Mathematically
Class ( x)  arg max (  g ( y ( x), c ))

k
k i
y k (x) is the classification of kth classifier, i is no. of classes

and g(y,c) is an indicator function defined as:
1 yc
g ( y, c)  
0 yc
Rokach, L. (2010). Ensemble-based classifiers. Artificial Intelligence Review,33(1-2), 1-39.

46
MV5 Framework
47
MV5 Example
• Classifiers training is performed on training data
• Suppose, each classifier predicted the following classes for a test

instance:
NB= 0, DT-GI=0, DT-IG= 1, KNN= 0, SVM= 1
Class 0: NB + DT-GI + KNN  3 votes

Class 1: DT-IG + SVM  2 votes
• Final output: Class 0
48
Performance comparison of MV5 for heart disease datasets
Acc Prec Rec F-M Acc Prec Rec F-M
Classifiers
Cleveland Dataset Eric Dataset
NB 77.23% 81.71% 71.94% 76.51% 68.90% 77.78% 57.61% 66.19%
DT-GI 75.91% 79.74% 72.00% 75.67% 71.77% 75.89% 67.01% 71.18%
DT-IG 73.60% 76.92% 70.07% 73.34% 75.12% 75.19% 75.00% 75.10%
SVM 78.22% 74.26% 86.14% 79.76% 78.95% 76.64% 73.33% 74.95%
KNN 64.36% 68.90% 58.99% 63.56% 65.55% 68.38% 61.96% 65.01%
MV5 85.23% 84.10% 86.68% 85.37% 82.22% 77.99% 74.00% 75.94%
SPECT Dataset SPECTF Dataset
NB 80.52% 76.36% 81.60% 78.90% 78.28% 23.64% 92.45% 37.65%
DT-GI 78.65% 48.08% 86.05% 61.69% 74.16% 38.71% 84.88% 53.17%
DT-IG 78.65% 51.85% 93.01% 66.58% 77.15% 41.18% 82.40% 54.91%
SVM 81.52% 36.36% 95.75% 52.71% 79.50% 47.27% 87.26% 61.32%
KNN 79.40% 81.10% 66.91% 73.32% 71.91% 36.36% 81.13% 50.22%
MV5 81.99% 81.21% 95.86% 87.92% 80.15% 55.56% 93.93% 69.82%
Statlog Dataset Acc = Accuracy Prec= Precision
NB 78.52% 82.00% 74.17% 77.89% Rec= Recall F-M = F-Measure
DT-GI 74.81% 79.29% 70.00% 74.35% NB = Naïve Bayes
DT-IG 73.33% 75.00% 71.05% 72.97% SVM = Support Vector Machine
SVM 78.52% 73.96% 89.74% 81.09% DT-IG = Decision tree using information gain
KNN 65.56% 68.67% 61.67% 64.98% DT-GI = Decision tree using gini index
MV5 80.52% 82.11% 90.92% 86.29% KNN = K Nearest Neighbor
49
Proposed Ensemble Model 1 - MV5
• Limitations
• Majority voting is the simplest way of combining
predictions
• More complex methods can be employed to improve
accuracy
50
AccWeight
51
Proposed Ensemble Model 2- AccWeight
• Improvement in MV5
• Weighted voting ensemble scheme instead of
majority voting
52
• Weighted Voting Ensemble using Accuracy Measure
(AccWeight)
• Five heterogeneous classifiers as used in MV5
• Naïve Bayes
• Decision Tree using Gini index
• Classifiers combination using weighted scheme

• High accuracy classifier will attain high weight and
vice versa
53
Weighted Voting Ensemble-AccWeight
The weighted voting ensemble will output the class which has
highest weight (Accuracy) associated with it. Mathematically
Class ( x)  arg max (  Acc( A ) * g ( y ( x), c ))

k
k k i
y k (x) is the classification of kth classifier, i is no. of classes

and g(y,c) is an indicator function defined as:
1 yc
g ( y, c)  
0 yc

54
AccWeight Framework
55
AccWeight Example
• Classifier training is performed on training data, calculate Accuracy
NB=70%, DT-GI= 75%, DT-IG=80%, KNN=82%, SVM=87%
• The resultant weights are as follows:

NB=0.7, DT-GI=0.75, DT-IG=0.8, KNN=0.82, SVM= 0.87
• Suppose, following classes are predicted for a test instance:

NB= 1, DT-GI=0, DT-IG= 1, KNN= 0, SVM= 1
Class 0: DT-GI + KNN  0.75+0.82  1.57

Class 1: NB+ DT-IG + SVM  0.7+0.8+0.87  2.37
56
Comparison between MV5 and AccWeight ensembles for Cleveland heart

disease dataset
Classifiers Accuracy Precision Recall F-Measure
MV5 85.23% 84.10% 86.68% 85.37%
AccWeight 86.82% 86.18% 87.27% 86.72%
57
Performance comparison of AccWeight for heart disease datasets
Classifiers
NB 77.23% 81.71% 71.94% 76.51% 68.90% 77.78% 57.61% 66.19%
DT-GI 75.91% 79.74% 72.00% 75.67% 71.77% 75.89% 67.01% 71.18%
DT-IG 73.60% 76.92% 70.07% 73.34% 75.12% 75.19% 75.00% 75.10%
SVM 78.22% 74.26% 86.14% 79.76% 78.95% 76.64% 73.33% 74.95%
KNN 64.36% 68.90% 58.99% 63.56% 65.55% 68.38% 61.96% 65.01%
AccWeight 86.82% 86.18% 87.27% 86.72% 83.12% 78.31% 76.60% 77.44%
NB 80.52% 76.36% 81.60% 78.90% 78.28% 23.64% 92.45% 37.65%
DT-GI 78.65% 48.08% 86.05% 61.69% 74.16% 38.71% 84.88% 53.17%
DT-IG 78.65% 51.85% 93.01% 66.58% 77.15% 41.18% 82.40% 54.91%
SVM 81.52% 36.36% 95.75% 52.71% 79.50% 47.27% 87.26% 61.32%
KNN 79.40% 81.10% 66.91% 73.32% 71.91% 36.36% 81.13% 50.22%
AccWeight 82.40% 81.32% 95.87% 87.99% 81.00% 71.71% 94.50% 81.54%
KNN 65.56% 68.67% 61.67% 64.98% DT-GI = Decision tree using gini index
AccWeight 83.04% 82.67% 91.31% 86.77% KNN = K Nearest Neighbor
58
• Limitations
• The biasness of accuracy results can be generated if
we have biased dataset
• Sometimes a single measure cannot capture the
quality of a good ensemble reliably
59
FmWeight
60
Proposed Ensemble Model 3- FmWeight
• Improvements in AccWeight
• Weighted voting ensemble using F-Measure
• An unbiased metric which assigns the weight to base
classifiers instead of accuracy measure
• Multiple objectives weighted voting such as precision
and recall
61
Proposed Ensemble Model 3- FmWeight
 Weighted Voting Ensemble using F-Measure
(FmWeight)
 Five heterogeneous classifiers as used in MV5 and
AccWeight
• Naïve Bayes
• Decision Tree using Gini index
 Classifiers combination using weighted voting

 Weighted vote based on F-Measure (Precision + Recall)
62
Weighted Voting Ensemble-FmWeight
The weighted voting ensemble will output the class which has
highest weight (F-Measure) associated with it. Mathematically
Class ( x)  arg max ( F  M ( yk ) * g ( yk ( x), ci ))

k
y k (x) is the classification of kth classifier, i is no. of output classes

g(y,c) is an indicator function defined as:
1 yc
g ( y, c)  
0 yc

63
FmWeight Ensemble Framework
64
FmWeight Example
• Classifier training is performed on training data, calculate F-Measure
NB=60%, DT-Gini= 70%, DT-info=80%, IBL=85%, SVM=90%

NB=0.6, DT-Gini=0.7, DT-Info=0.8, IBL=0.85, SVM= 0.9

NB= 0, DT-Gini=0, DT-Info= 1, IBL= 0, SVM= 1
Class 0: NB + DT-Gini + IBL  0.6+0.7+0.85 2.15

Class 1: DT-Info + SVM  0.8+0.9  1.7
65
Comparison between MV5, AccWeight & FmWeight Ensembles for
Cleveland heart disease dataset
MV5 85.23% 84.10% 86.68% 85.37%
AccWeight 86.82% 86.18% 87.27% 86.72%
FmWeight 87.37% 87.50% 88.27% 87.90%
66
Performance comparison of FmWeight for heart disease datasets
Classifiers
NB 77.23% 81.71% 71.94% 76.51% 68.90% 77.78% 57.61% 66.19%
DT-GI 75.91% 79.74% 72.00% 75.67% 71.77% 75.89% 67.01% 71.18%
DT-IG 73.60% 76.92% 70.07% 73.34% 75.12% 75.19% 75.00% 75.10%
SVM 78.22% 74.26% 86.14% 79.76% 78.95% 76.64% 73.33% 74.95%
kNN 64.36% 68.90% 58.99% 63.56% 65.55% 68.38% 61.96% 65.01%
FmWeight 87.37% 87.50% 88.27% 87.90% 84.19% 83.75% 77.53% 80.52%
NB 80.52% 76.36% 81.60% 78.90% 78.28% 23.64% 92.45% 37.65%
DT-GI 78.65% 48.08% 86.05% 61.69% 74.16% 38.71% 84.88% 53.17%
DT-IG 78.65% 51.85% 93.01% 66.58% 77.15% 41.18% 82.40% 54.91%
SVM 81.52% 36.36% 95.75% 52.71% 79.50% 47.27% 87.26% 61.32%
kNN 79.40% 81.10% 66.91% 73.32% 71.91% 36.36% 81.13% 50.22%
FmWeight 83.75% 81.40% 95.99% 88.09% 82.73% 72.67% 96.33% 82.84%
kNN 65.56% 68.67% 61.67% 64.98% DT-GI = Decision tree using gini index
FmWeight 86.82% 87.50% 93.27% 90.29% KNN = k Nearest Neighbor
67
BagMoov
68
Proposed Ensemble Model 4- BagMoov
• Improvements in FmWeight
• Bagging is enhanced with weighted metric
• F-Measure is used to assign weight
69
• Bagging Approach with Multi-objective Weighted
voting Ensemble Scheme (BagMoov)
• Previous models focused on heart disease diagnosis

only
• Now we have focused on multiple diseases diagnosis for

selection of classifiers
70
• Performed literature survey of almost 500 research

papers published in last five years relating to multiple
diseases including, breast cancer, liver, diabetes and
hepatitis
• Top 5 classifiers are selected based on high accuracy

and diversity for diagnosis of multiple diseases
71
• Based on literature, following five classifiers are selected
to make an ensemble model:
• Naïve Bayes (NB)
• Support Vector Machine (SVM)
• K Nearest Neighbor (KNN)
• Linear Regression (LR)
• Linear Discriminant Analysis (LDA)
• Classifiers combination using proposed Bagging

approach with multi-objective weighted voting
72
• In addition to NB, SVM and KNN that are used in previous frameworks;
Now, we have two more classifiers:
• Linear Regression
y  w0  w1 x
A straight line is used to fit the data
x is given variable and y is predicted value,w0 is y-intercept and w1 is slope of line
Least square method: choose the best fitting straight line ( values of w0 and w1)
• Establishes a functional relationship between set of variables -> Resolves

Naïve Bayes issue
• Achieves optimal results, requires very small amount of training data
• Limits the prediction of numeric output -> Resolved by KNN and LDA
• Sensitive to outliers -> Preprocessing
Campbell MJ. Blackwell BMJ Books, 2006. Statistics at Square Two. 2nd Ed.
73
• Linear Discriminant Analysis
D  b1 X 1  b2 X 2  b3 X 3  ........  bk X k  c
D=Discriminant score, b’s=Discriminant coefficient or weight
X’s= Input variables, c= Constant
• Predicts nominal class labels

• Easier to train, low variance, more efficient
• Correlation between variables -> Resolve Naïve Bayes issue
• Sensitive to overfit -> Resolved by SVM
J. Vander Sloten, P. Verdonck, M. Nyssen, J. Haueisen (Eds.): ECIFMBE 2008, IFMBE Proceedings 22, pp. 389–392, 2008
74
Proposed Bagging Approach with Multi-objective
Weighted Voting
BagMoov (T,M) // T is original training set of N samples
For each m=1,2,…..M //M is no. of base classifiers to be learned
Tm  Sample  With  Re placement (T , N )

//Lb is base classifier learning algorithm,
y m  Lb (Tm ) y m is the trained base classifier
endfor
Class ( x)  arg max ( F  M ( yk ) * g ( yk ( x), ci ))

k
y k (x) is the classification of kth classifier, i is 0 and 1, and g(y,c) is an indicator function defined as:
1 yc
g ( y, c)  
0 yc
Sample-With-Replacement (T,N)
S={ }
For i=1,2,……N
r=random_integer (1,N)
Add T[r] to S
Return S
75
BagMoov Framework
76
BagMoov Example
• Classifiers training is performed on training data, calculate F-Measure
NB=60%, LR= 70%, LDA=80%, KNN=85%, SVM=90%
• The weights are as follows:

NB=0.6, LR=0.7, LDA=0.8, KNN=0.85, SVM= 0.9

NB= 0, LR=0, LDA= 1, KNN= 0, SVM= 1
Class 0: NB + LR + KNN  0.6+0.7+0.85  2.15

Class 1: LDA + SVM  0.8+0.9  1.7
77
 Benchmark datasets taken from UCI data repository
• 5 heart disease datasets
• 4 breast cancer datasets
• 2 liver disease datasets
• 2 diabetes datasets
• 1 hepatitis dataset
78
Comparison between MV5, AccWeight, FmWeight & BagMoov

Ensembles for Cleveland heart disease dataset
MV5 85.23% 84.10% 86.68% 85.37%
AccWeight 86.82% 86.18% 87.27% 86.72%
FmWeight 87.37% 87.50% 88.27% 87.90%
BagMoov 88.83% 89.01% 89.17% 89.09%
79
Experimental Results- Heart disease
Performance comparison of BagMoov for heart disease datasets
Classifiers
NB 77.23% 81.71% 71.94% 76.51% 68.90% 77.78% 57.61% 66.19%
LR 83.50% 88.41% 77.70% 82.71% 77.99% 88.89% 64.13% 74.51%
LDA 65.68% 68.29% 62.59% 65.32% 69.41% 64.00% 73.33% 68.35%
SVM 78.22% 74.26% 86.14% 79.76% 78.95% 76.64% 73.33% 74.95%
KNN 64.36% 68.90% 58.99% 63.56% 65.55% 68.38% 61.96% 65.01%
BagMoov 88.83% 89.01% 89.17% 89.09% 85.16% 93.29% 78.38% 85.19%
NB 80.52% 76.36% 81.60% 78.90% 78.28% 23.64% 92.45% 37.65%
LR 83.15% 38.18% 94.81% 54.44% 78.28% 68.38% 61.96% 65.01%
LDA 83.52% 36.36% 95.75% 52.71% 68.60% 47.27% 87.26% 61.32%
SVM 81.52% 36.36% 95.75% 52.71% 79.50% 47.27% 87.26% 61.32%
KNN 79.40% 81.10% 66.91% 73.32% 71.91% 36.36% 81.13% 50.22%
BagMoov 84.92% 83.27% 96.23% 89.28% 83.28% 75.27% 96.70% 84.65%
Statlog Dataset
Acc = Accuracy Prec= Precision
LR 82.59% 87.33% 76.67% 81.65% NB = Naïve Bayes
LDA 68.15% 64.00% 73.33% 68.35% SVM= Support Vector Machine
SVM 78.52% 73.96% 89.74% 81.09% LR = Linear Regression
LDA = Linear Discriminant Analysis
KNN 65.56% 68.67% 61.67% 64.98%
80 KNN = K Nearest Neighbor
BagMoov 87.07% 92.00% 95.17% 93.56%
Experimental Results- Breast Cancer
Performance comparison of BagMoov for breast cancer datasets
Classifiers UMC Dataset WPBC Dataset
NB 68% 78.9% 75.04% 76.92% 72.76% 36.36% 16.5% 22.70%
SVM 70.31% 73.76% 89.62% 80.92% 76.29% 67.65% 69.99% 68.80%
LR 71.33% 74.16% 91.12% 81.77% 78.87% 63.02% 46.5% 53.51%
LDA 71.05% 73.75% 91.60% 81.71% 72.95% 45.46% 85.55% 59.37%
KNN 62.22% 69.76% 81.6% 75.22% 66.13% 28.67% 15% 19.70%

BagMoov 81.61% 79.17% 92.30% 85.23% 79.11% 85.01% 85.99% 85.49%
WDBC Dataset WBC Dataset
NB 93.45% 95.03% 94.67% 94.85% 96.7% 91.95% 99.58% 95.61%
SVM 96.84% 96.27% 99.16% 97.69% 96.86% 95.02% 96.28% 95.65%
LR 95.26% 93.78% 99.44% 96.53% 95.85% 96.26% 91.68% 93.91%
LDA 79.63% 78.97% 93.78% 85.74% 70.53% 95.24% 15.81% 27.12%
KNN 78.04% 77.67% 91.31% 83.94% 63.09% 45.73% 36.63% 40.68%
BagMoov 96.99% 97.04% 99.50% 98.25% 96.98% 97.01% 99.65% 98.31%
81
Experimental Results- Diabetes
Performance comparison of BagMoov for diabetes datasets

Classifiers
Pima Indian Diabetes Dataset Biostat Diabetes Dataset
NB 71.09% 81.15% 72.6% 76.64% 85.61% 51.45% 49.76% 50.59%
SVM 76.95% 79.22% 88% 83.38% 91.57% 88.83% 50.43% 64.34%
LR 77.08% 79.15% 88.4% 83.52% 92.07% 87.67% 55.19% 67.74%
LDA 57.94% 82.98% 46.8% 59.85% 84.88% 74.88% 75% 74.94%
KNN 69.94% 75.96% 79% 77.45% 85.37% 54.55% 60.1% 57.19%
BagMoov 82.95% 79.50% 89.98% 84.42% 92.55% 89.67% 84.32% 86.91%
82
Experimental Results- Liver Disease
Performance comparison of BagMoov for liver disease datasets

Classifiers
ILPD Dataset BUPA liver disease Dataset
NB 68.39% 46.84% 75.35% 57.77% 60.02% 63.8% 72.59% 67.91%
SVM 71.36% 68.04% 67.67% 67.85% 67.51% 67.18% 86.02% 75.44%
LR 71.36% 50.98% 55.44% 53.12% 67.52% 69.41% 79.06% 73.92%
LDA 71.36% 63.10% 62.97% 63.03% 65.10% 62.97% 63.99% 63.47%
KNN 65.2% 68.66% 64.15% 66.33% 63.78% 67.21% 73.92% 70.41%
BagMoov 78.01% 79.42% 78.56% 78.99% 76.09% 76.62% 85.67% 80.89%
83
Experimental Results- Hepatitis
Performance comparison of BagMoov for hepatitis dataset
NB 83% 66.67% 36.67% 47.32%
SVM 85% 66.67% 54.17% 59.77%
LR 84.38% 70.17% 55% 61.67%
LDA 83.75% 68.42% 40% 50.49%
KNN 74.04% 71.43% 69.17% 70.28%
BagMoov 86.57% 81.53% 80.89% 81.21%
84
• Limitations
• Single layer ensemble model
• Experimentation showed no further increase in
accuracy as the number of classifiers are increased
• Multi-layer ensemble models may result in higher
accuracy
85
HM-BagMoov
86
Proposed Ensemble Model 5- HM-BagMoov
• Improvement in BagMoov
• Multi-layer ensemble framework in order to further
improve disease diagnosis accuracy
87
Proposed Ensemble Model 5- HM-BagMoov
 Hierarchical Multi-level classifiers Bagging with
Multi-objective Weighted voting (HM-BagMoov)
 Optimal combination of single and ensemble classifiers
 Arrangement of classifiers in layered structure
 Combining of classifiers outputs using proposed

BagMoov ensemble approach
88
Layered Structure
• Taxonomy structure, improves classification efficiency and
accuracy
• Computing performance at each level, reduces the risk of

making an error in the top down classification process
• Different classifiers for different layers, different features for

each layer; classification tasks are more refined
• Scalable for hierarchical training and classification
Zolfaghar, K., Verbiest, N., Agarwal, J., Meadem, N., Chin, S. C., Roy, S. B., ... & Reed, L. (2013). Predicting risk-of-readmission for
congestive heart failure patients: A multi-layer approach. arXiv preprint arXiv:1306.2094
89
• So far we have used single classifiers in proposed
frameworks
• For first layer, same classifiers as used in BagMoov

framework
• For second layer, we focused on ensemble classifiers as

experimentation on BagMoov showed no further
increase in accuracy at layer 1
• Literature survey of ensemble classifiers was carried out
90
• From literature, top two ensembles are selected
based on highest accuracy and diversity for multiple
diseases diagnosis
• Artificial Neural Network Ensemble (ANN)
• Random Forest (RF)
91
• Artificial Neural Network Ensemble
x0 w0 - k
x1 w1
 f output y
xn wn
For Example
n
Input weight weighted y  sign( wi xi   k )
Activation i 0
vector x vector w sum
function
• Very low prediction error

• Well-suited for continuous-valued inputs and outputs -> Resolve Naïve
Bayes issue
• Suitable for prediction and disease diagnosis
Er, O., Cetin, O., Bascil, M. S., & Temurtas, F. (2016). A Comparative Study on Parkinson's Disease Diagnosis Using Neural
Networks and Artificial Immune System. Journal of Medical Imaging and Health Informatics, 6(1), 264-268.
92
• Random Forest
n n is no. of classes
gini(D) 1  p 2j
j 1 Gini index is splitting criteria,
select attribute with smallest gini
• Improves prediction accuracy compared to single tree

• Deals really well with uneven data sets
• Handles both continuous and nominal attribute -> Resolve NN issue
• Fast to build and faster to predict
Ani, R., Augustine, A., Akhil, N. C., & Deepa, O. S. (2016). Random Forest Ensemble Classifier to Predict the Coronary Heart
Disease Using Risk Factors. In Proceedings of the International Conference on Soft Computing Systems (pp. 701-710). Springer
93
Architecture of HM-BagMoov
• Layer 1
• Five heterogeneous classifiers as used in BagMoov
• Naïve Bayes
• Linear Regression
• Linear Discriminant Analysis
• Support Vector Machines
94
• Layer 2
• Output of layer-1 classifiers is combined using
proposed weighted bagging ensemble approach
• Two ensemble classifiers are added
• Artificial Neural Network Ensemble
• Random Forest
95
• Layer 3
• Final result is obtained
• Final output of the proposed ensemble is labeled with
the class that has highest weighted vote
96
HM-BagMoov Framework
97
HM-BagMoov Example
• Classifiers training is performed on training data, calculate F-Measure
NB=60%, LDA= 70%, LR=80%, KNN=85%, SVM=82%, ANN=85%, RF= 87%

NB=0.6, LDA=0.7, LR=0.8, KNN=0.85, SVM= 0.82, ANN=0.85, RF=0.87
• Layer 1: Class output for a test instance

NB= 0, LDA=0, LR= 1, KNN= 0, SVM= 1
Class 0: NB + LDA + KNN  0.6+0.7+0.85  2.15
Class 1: LR + SVM  0.8+0.82  1.62
Layer 1 output: 0
• Layer 2: class output for same test instance
ANN=1, RF=0, Class 0
Class 0: RF + Class 0  0.87+2.15  3.02
Class 1: ANN  0.85
Layer 2 output: 0
• Layer 3: Class 0
98
Comparison between MV5, AccWeight, FmWeight, BagMoov &

HM-BagMoov Ensembles for Cleveland heart disease dataset
MV5 85.23% 84.10% 86.68% 85.37%
AccWeight 86.82% 86.18% 87.27% 86.72%
FmWeight 87.37% 87.50% 88.27% 87.90%
BagMoov 88.83% 89.01% 89.17% 89.09%
HM-BagMoov 91.99% 91.31% 90.19% 90.75%
99
Single Layer vs Layered Approach
Comparison of Single layer vs multi layer ensemble for Cleveland
heart disease dataset
Technique Accuracy Precision Recall F-Measure
Dataset
Single layer Ensemble 84.16% 82.07% 74.82% 78.27%

Cleveland
HM-BagMoov 91.99% 91.31% 90.19% 90.75%

Eric
HM-BagMoov 89.82% 94.74% 85.74% 90.01%

SPECT
HM-BagMoov 88.77% 87.33% 97.67% 92.21%

SPECTF
HM-BagMoov 89.21% 83.54% 97.10% 89.81%

Statlog
HM-BagMoov 90.93% 94.67% 96.50% 95.58%
100
Classifiers
NB 77.23% 81.71% 71.94% 76.51% 68.90% 77.78% 57.61% 66.19%
LR 83.50% 88.41% 77.70% 82.71% 77.99% 88.89% 64.13% 74.51%
LDA 65.68% 68.29% 62.59% 65.32% 69.41% 64.00% 73.33% 68.35%
SVM 78.22% 74.26% 86.14% 79.76% 78.95% 76.64% 73.33% 74.95%
kNN 64.36% 68.90% 58.99% 63.56% 65.55% 68.38% 61.96% 65.01%
RF 69.64% 84.76% 51.80% 64.30% 69.86% 90.60% 43.48% 58.76%
ANN 79.21% 79.88% 78.42% 79.14% 76.08% 80.34% 70.65% 75.19%
HM-BagMOOV 91.99% 91.31% 90.19% 90.75% 89.82% 94.74% 85.74% 90.01%
NB 80.52% 76.36% 81.60% 78.90% 78.28% 23.64% 92.45% 37.65%
LR 83.15% 38.18% 94.81% 54.44% 78.28% 68.38% 61.96% 65.01%
LDA 83.52% 36.36% 95.75% 52.71% 68.60% 47.27% 87.26% 61.32%
SVM 81.52% 36.36% 95.75% 52.71% 79.50% 47.27% 87.26% 61.32%
kNN 79.40% 81.10% 66.91% 73.32% 71.91% 36.36% 81.13% 50.22%
RF 79.40% 59.09% 87.00% 70.38% 79.40% 77.76% 71.60% 74.55%
ANN 79.78% 50.91% 87.26% 64.30% 78.65% 50.91% 85.85% 63.92%
HM-BagMOOV 88.77% 87.33% 97.67% 92.21% 89.21% 83.54% 97.10% 89.81%
Statlog Dataset
NB 78.52% 82.00% 74.17% 77.89% Acc = Accuracy Prec= Precision
LR 82.59% 87.33% 76.67% 81.65% Rec= Recall F-M = F-Measure
LDA 68.15% 64.00% 73.33% 68.35% NB = Naïve Bayes, SVM = Support Vector
Machine, LR = Linear Regression
SVM 78.52% 73.96% 89.74% 81.09%
LDA = Linear Discriminant Analysis
kNN 65.56% 68.67% 61.67% 64.98% kNN = k Nearest Neighbor
RF 71.11% 81.33% 58.33% 67.94% RF= Random Forest
ANN 78.15% 79.33% 76.67% 77.98% ANN= Artificial Neural Network
101 HM-BagMOOV 90.93% 94.67% 96.50% 95.58%
Performance comparison of HM-BagMoov with other proposed ensembles
for heart disease datasets
Ensembles
MV5 85.23% 84.10% 86.68% 85.37% 82.22% 77.99% 74.00% 75.94%

AccWeight 86.82% 86.18% 87.27% 86.72% 83.12% 78.31% 76.60% 77.44%
FmWeight 87.37% 87.50% 88.27% 87.90% 84.19% 83.75% 77.53% 80.52%
BagMoov 88.83% 89.01% 89.17% 89.09% 85.16% 93.29% 78.38% 85.19%
HM-BagMoov 91.99% 91.31% 90.19% 90.75% 89.82% 94.74% 85.74% 90.01%

MV5 81.99% 81.21% 95.86% 87.92% 80.15% 55.56% 93.93% 69.82%
AccWeight 82.40% 81.32% 95.87% 87.99% 81.00% 71.71% 94.50% 81.54%
FmWeight 83.75% 81.40% 95.99% 88.09% 82.73% 72.67% 96.33% 82.84%
BagMoov 84.92% 83.27% 96.23% 89.28% 83.28% 75.27% 96.70% 84.65%
HM-BagMoov 88.77% 87.33% 97.67% 92.21% 89.21% 83.54% 97.10% 89.81%
Statlog Dataset
MV5 80.52% 82.11% 90.92% 86.29%
AccWeight Acc = Accuracy Prec= Precision
83.04% 82.67% 91.31% 86.77%
Rec= Recall F-M = F-Measure
FmWeight 86.82% 87.50% 93.27% 90.29%
BagMoov 87.07% 92.00% 95.17% 93.56%
HM-BagMoov
102 90.93% 94.67% 96.50% 95.58%
Performance comparison of HM-BagMoov with multi-layer ensembles for heart disease datasets
Ensembles
Majority voting 82.84% 87.80% 76.98% 82.04% 77.51% 87.18% 65.22% 74.62%
Weighting 84.16% 72.95% 74.21% 73.57% 81.50% 85.55% 70.99% 77.59%
Bagging 85.16% 82.68% 76.10% 79.25% 81.82% 89.74% 71.74% 79.74%
AdaBoost 79.21% 83.54% 74.10% 78.54% 77.03% 80.34% 72.83% 76.40%
Stacking 71.90% 75% 69.98% 72.40% 70.16% 68.81% 69.52% 69.16%
HM-BagMoov 91.99% 91.31% 90.19% 90.75% 89.82% 94.74% 85.74% 90.01%

Majority voting 83.52% 36.36% 95.75% 52.71% 79.78% 49.09% 95.11% 64.75%
Weighting 83.99% 32.50% 93.99% 48.29% 80.03% 45.45% 95.50% 61.59%
Bagging 85.77% 52.73% 95.75% 68.01% 84.03% 55.45% 96.19% 70.35%
AdaBoost 82.92% 36.36% 96.23% 52.78% 78.28% 45.45% 86.79% 59.66%
Stacking 82.85% 21.43% 54.17% 30.71% 77.08% 79.15% 88.4% 83.52%
HM-BagMoov 88.77% 87.33% 97.67% 92.21% 89.21% 83.54% 97.10% 89.81%
Statlog Dataset
Majority voting 82.59% 90.67% 72.50% 80.57%
Weighting 85.55% 92.67% 77.50% 84.40% Acc = Accuracy Prec= Precision
Bagging 85.93% 93.67% 79.50% 86.01% Rec= Recall F-M = F-Measure
AdaBoost 78.89% 83.33% 73.33% 78.01%
Stacking 72.38% 73.75% 75.55% 74.64%
103
HM-BagMoov 90.93% 94.67% 96.50% 95.58%
State of the art comparison of HM-BagMoov for Cleveland heart
disease dataset
Reference Year Technique Accuracy Precision Recall
Yang, J. et al. [78 ] 2015 Framingham risk score 64.65% - -

Linear discriminant analysis 65.66%
Decision tree 71.72%

Logistic regression 71.72%
Neural Network 73.74%
Peter, J. et al. [79] 2015 Naïve Bayes continuous 67.4% - -
variable
Naïve Bayes discrete variable 78.2%
Kiruthika, C. et al. [80] 2014 K-Means clustering 80% - -

Fuzzy C Means clustering 78%
Shouman, M., et al. [4] 2013 Random forest 82.1% 75.6% 79.1%
Shouman, M., et al. [46] K mean clustering with Naïve
2012 76.59% 69.93% 78.62%
Bayes algorithm
Proposed Technique HM-BagMoov 91.99% 91.31% 90.19%
104
Performance comparison of HM-BagMoov for breast cancer datasets
Classifiers UMC Dataset WPBC Dataset
NB 68% 78.9% 75.04% 76.92% 72.76% 36.36% 16.5% 22.70%
SVM 70.31% 73.76% 89.62% 80.92% 76.29% 67.65% 69.99% 68.80%
LR 71.33% 74.16% 91.12% 81.77% 78.87% 63.02% 46.5% 53.51%
LDA 71.05% 73.75% 91.60% 81.71% 72.95% 45.46% 85.55% 59.37%
kNN 62.22% 69.76% 81.6% 75.22% 66.13% 48.67% 75% 59.03%

RF 70.28% 73.9% 74.14% 74.02% 76.77% 66.67% 71.6% 69.05%
ANN 65.73% 30.59% 80.60% 44.35% 78.28% 87.42% 48.94% 62.75%
HM-BagMoov 83.45% 80.19% 95.57% 87.20% 80.34% 90.00% 86.51% 88.22%
NB 93.45% 95.03% 94.67% 94.85% 96.7% 91.95% 99.58% 95.61%
SVM 96.84% 96.27% 99.16% 97.69% 96.86% 95.02% 96.28% 95.65%
LR 95.26% 93.78% 99.44% 96.53% 95.85% 96.26% 91.68% 93.91%
LDA 79.63% 78.97% 93.78% 85.74% 70.53% 95.24% 15.81% 27.12%
kNN 78.04% 77.67% 91.31% 83.94% 63.09% 45.73% 36.63% 40.68%
RF 88.05% 69.81% 98.88% 81.84% 92.85% 87.14% 95.85% 91.29%
ANN 96.13% 92.92% 98.04% 95.41% 96.42% 96.27% 96.51% 91.29%
105
HM-BagMoov 97.56% 98.11% 99.56% 98.83% 98.11% 98.78% 99.85% 99.31%
Performance comparison of HM-BagMoov with multi-layer ensembles for
breast cancer datasets
Classifiers
UMC Dataset WPBC Dataset
Majority voting 68.38% 70.00% 91.53% 79.33% 75.32% 68.01% 51.91% 58.88%
Weighting 71.05% 45.46% 85.55% 59.37% 78.85% 77.95% 62.91% 69.63%
Bagging 72.95% 73.75% 94.60% 82.88% 79.24% 80.00% 70.00% 74.67%
AdaBoost 64.34% 68.24% 79.60% 73.48% 79.80% 92.72% 58.30% 71.59%
Stacking 70.28% 53.85% 24.71% 33.87% 70.98% 51.47% 41.18% 45.75%
HM-BagMoov 83.45% 80.19% 95.57% 87.20% 80.34% 90.00% 86.51% 88.22%

Majority voting 92.61% 89.15% 89.44% 89.29% 96.71% 94.38% 95.14% 94.76%
Weighting 96.01% 94.23% 92.50% 93.36% 96.98% 93.28% 95.01% 94.14%

Bagging 96.99% 94.60% 95.04% 94.82% 97.57% 97.13% 97.68% 97.40%
AdaBoost 96.13% 94.34% 87.20% 90.63% 95.85% 92.95% 94.38% 93.66%

Stacking 95.99% 85.83% 95.41% 90.37% 94.71% 87.14% 85.85% 86.49%
HM-BagMoov 97.56% 98.11% 99.56% 98.83% 98.11% 98.78% 99.85% 99.31%

106
State of the art comparison of HM-BagMoov for WBC dataset
Sørensen, K.P. et al. [81] 2015 Long term coding RNA 92% 90% 65%
Zand, H.K.K. [82] 2015 Naïve Bayes 84.5% 70% 57%

Artificial Neural Net 86.5% 83% 52%
Ensemble
C4.5 86.7% 80% 56%
Chaurasia, V. et al. [83] 2014 RepTree 73.7% 92%
RBF network 77.4% 88.6%
Simple logistic 74.47% 76.2% 92.5%
Chaurasia, V. et al. [84] 2014 BF tree 96% 97%
IBK 95.8% 98%
SMO 96.2% 97.1% 97.1%
Support Vector Machine 75.00% - -
K, A.A., et. Al. [85] 2013
Random forest 75.00% - -
Proposed technique HM-BagMoov 98.11% 98.78% 99.85%
107
Performance comparison of HM-BagMoov for diabetes datasets

Classifiers
NB 71.09% 81.15% 72.6% 76.64% 85.61% 51.45% 49.76% 50.59%
SVM 76.95% 79.22% 88% 83.38% 91.57% 88.83% 50.43% 64.34%
LR 77.08% 79.15% 88.4% 83.52% 92.07% 87.67% 55.19% 67.74%
LDA 57.94% 82.98% 46.8% 59.85% 84.88% 74.88% 75% 74.94%
kNN 69.94% 75.96% 79% 77.45% 85.37% 54.55% 60.1% 57.19%
RF 83.10% 85.22% 89% 87.07% 94.11% 89.83% 70.43% 78.95%
ANN 82.74% 83.84% 88.80% 86.25% 93.83% 89.34% 68.33% 77.43%
HM-BagMoov 85.21% 88.65% 92.6% 90.58% 95.07% 90.31% 99.88% 94.85%
108
diabetes datasets
Classifiers
Majority voting 76.30% 50.00% 80.40% 61.66% 91.07% 79.54% 48.33% 60.13%
Weighting 77.00% 65.54% 85.55% 74.21% 92.24% 83.55% 45.76% 59.13%
Bagging 77.99% 75.96% 91.23% 82.89% 94.29% 86.83% 62.48% 72.67%
AdaBoost
76.43% 52.99% 89.00% 66.42% 88.83% 85.79% 43.33% 57.58%
Stacking
74.61% 69.99% 75.39% 72.59% 87.78% 81.88% 60.11% 69.33%
HM-BagMoov 85.21% 88.65% 92.6% 90.58% 95.07% 90.31% 99.88% 94.85%
109
State of the art comparison of HM-BagMoov for Biostat dataset
Kandhasamy, J.P. et al. [86] 2015 J48 73.82% 59.7% 81.4%
KNN 73.17% 53.7% 83.6%
SVM 73.34% 53.84% 73.39%
Random forest 71.74% 53.81% 80.4%
Bashir, S. et al. [87] 2014 Majority voting 74.09% 89.4% 45.52%
AdaBoost 74.22% 84.2% 55.6%
Bayesian Boosting 73.18% 82.6% 55.6%
Stacking 68.23% 76% 53.73%
Bagging 74.48% 81.4% 61.5%
- -
Gandhi, K.K. et al. [88] 2014 F-score FS+ SVM 75%
Tapak, L. et al. [89] 2013 Logistic regression 76.3% 13.3% 99.9%
Linear discriminant analysis 71% .6% 99.8%
Fuzzy c-mean 67.8% 33% 90.1%
Neural network 75.1% 8.4% 99.8%
Random forest 71.7% 8.1% 99.8%
Karthikeyani, V. et al. [90] 2012 Discriminant analysis 76% - -
110
Performance comparison of HM-BagMoov for liver disease datasets

Classifiers
NB 68.39% 46.84% 75.35% 57.77% 60.02% 63.8% 72.59% 67.91%
SVM 71.36% 68.04% 67.67% 67.85% 67.51% 67.18% 86.02% 75.44%
LR 71.36% 50.98% 55.44% 53.12% 67.52% 69.41% 79.06% 73.92%
LDA 71.36% 63.10% 62.97% 63.03% 65.10% 62.97% 63.99% 63.47%
kNN 65.2% 68.66% 64.15% 66.33% 63.78% 67.21% 73.92% 70.41%
RF 76.70% 70.17% 75% 72.50% 73.42% 76.67% 87.17% 81.58%
ANN 78.44% 74.13% 69.34% 71.65% 72.46% 75.69% 88.00% 81.38%
HM-BagMoov 82.7% 83.9% 81.76% 82.82% 80.16% 78.81% 89.52% 83.82%
111
liver disease datasets
Classifiers
Majority voting 71.53% 69.04% 62.99% 65.87% 71.88% 45.52% 81.00% 58.28%
Weighting 71.90% 75% 71.00% 72.94% 67.01% 59.99% 75.98% 67.04%
Bagging 77.87% 76.59% 66.92% 76.02% 66.72% 82.99% 73.97%

71.42%
AdaBoost 66.55% 71.49% 59.34% 64.85% 68.41% 53.79% 79.00% 64.00%
Stacking 62.03% 54.67% 56.55% 55.59% 63.98% 50.11% 74.86% 60.03%
HM-BagMoov 82.7% 83.9% 81.76% 82.82% 80.16% 78.81% 89.52% 83.82%
112
State of the art comparison of HM-BagMoov for ILPD dataset
J48 68.7%
MLP 68.2%
Gulia, A. et al. [91] 2015 SVM 71.3% - -
RandomForest 70.3%
BayesNet 67.2%
Naïve Bayes 53.9% 37.4% 75.2%
Decision tree 69.4% 73.1% 35.2%
Jin, H. et al. [29] 2014
Multi-layer perception 67.9% 72.9% 30.3%
K nearest neighbor 65.3% 72.7% 46.7%
CART 67.82%
Sug, H. [92] 2012 - -
C4.5 66.47%
NBC 55.59% 71.03% 37.5%
C4.5 55.94% 50.34% 60%
Ramana, B.V. et al. [33] 2011 Back propagation 66.66% 51.03% 78%
K-NN 57.97% 0% 1%
SVM 62.6% 55.86% 67.5%
Naïve Bayes 55% 39.7%
Karthik, S. et al. [34] 2011 -
RBF network 61.5% 46.6%
Proposed technique
113
HM-BagMoov 82.7% 83.9% 81.76%
Performance comparison of HM-BagMoov for hepatitis dataset
NB 83% 66.67% 36.67% 47.32%
SVM 85% 66.67% 54.17% 59.77%
LR 84.38% 70.17% 55% 61.67%
LDA 83.75% 68.42% 40% 50.49%
kNN 74.04% 69.43% 69.17% 69.29%
RF 85.12% 72.21% 70.92% 71.56%
ANN 85.77% 70.80% 71.13% 70.96%
HM-BagMoov 87.04% 83.27% 81.67% 82.46%
114
hepatitis datasets
Majority voting 77.12% 74.26% 60.63% 66.76%
Weighting 79.50% 75.63% 68.99% 72.16%
Bagging 83.55% 76.60% 70.99% 73.69%
AdaBoost 80.12% 75.08% 67.50% 71.09%
Stacking 79.00% 67.70% 69.91% 68.79%
HM-BagMoov 87.04% 83.27% 81.67% 82.46%
115
State of the art comparison of HM-BagMoov for hepatitis dataset
Artificial neural network 69% 77.3% 50.6%
Houby, E.M.F. [93] 2015 Associative classification 81.3% 62.5% 73.4%
Decision tree 73.3% 68.53% 74.3%
Bayes.NaiveBayes 84%
Bayes.BayesNet 81%
Bayes.NaiveBayes
84%
Karthikeyan, T. et al. [38 ] 2013 Updatable - -
J48 83%
Random forest 83%
Multilayer perception 83%
KNN 70.29%
Naïve Bayes 66.94%
Neshat, M. et al. [94] 2012 SVM 65.22% - -
FDT 61.49%
CBR-PSO 77.16%
SVM 83.12%
Kumar. M.V. et al. [40] 2012 - -
SVM and Wrapper method 74.55%
116
Processing Time
• The performance of HM-BagMoov is also compared wrt
time with ensemble approaches
• Comparison indicates that HM-BagMoov is efficient for

disease diagnosis in terms of processor time utilized
117
Processing Time
Comparison of HM-BagMoov with other ensemble classifiers
Processing Time
120
100
Cleveland
Eric
80 Statlog
SPECT
Time (ms)
SPECTF
60
PIMA
Diabetes
40 UMC
WBC
ILPD
20 BUPA
Hepatitis
0
Majority Voting AdaBoost Bagging HM-BagMoov
Ensemble Classifiers
118
Stopping Criteria
 The stopping criteria is based on:
• No. of classifiers
• No. of layers
• Processing time
119
Stopping Criteria
 If more no. of classifiers added then accuracy decreases
 Addition of two classifiers at layer 1 (9-classifiers)
 Perceptron and Polynomial regression
 Further addition of two classifiers at layer 2 (11-classifiers)
 SVM ensemble and Decision tree ensemble
 Cleveland heart disease dataset
Comparison of HM-BagMoov with 9 and 11 classifiers based approach

11-classifiers approach 79.01% 75.34% 69.08% 72.07%
9-classifiers approach 86.19% 82.93% 79.14% 80.99%

7- classifiers approach
91.99% 91.31% 90.19% 90.75%
(HM-BagMoov)
120
Stopping Criteria
 If more no. of layers added then accuracy decreases
 Addition of one more layer (4-layered approach)
 SVM ensemble and Decision tree ensemble
Comparison of HM-BagMoov with 4 layered approach
4-layered approach 87.18% 83.54% 80.58% 82.03%
3-layered approach
91.99% 91.31% 90.19% 90.75%
(HM-BagMoov)
121
Stopping Criteria
 Processing time increases if more classifiers or layers
added
Time comparison of HM-BagMoov with other approaches

Technique Time (ms)
11-classifiers
37.45
approach
9-classifiers
36.98
approach
4-layered approach 35.99
HM-BagMoov 35.10
122
Analysis of Proposed Technique
• An optimal model of class diversity
• Consistently produced the highest accuracy for all
diseases/datasets
• Each classifier in ensemble framework has diverse set of
qualities
• Complement and overcome the limitations of each other
• Make an accurate ensemble framework for disease
diagnosis
• The framework is not limited to attributes or records
• The framework can be used for any disease diagnosis
123
IntelliHealth: An Intelligent Application for
Disease Diagnosis
124
IntelliHealth
• A complete medical decision support application for
• Data acquisition and preprocessing
• Disease diagnosis and report generation
125
IntelliHealth Architecture
126
Users of IntelliHealth
• Three Users
• Each user has its login ID and password
• Admin/ I.T. Staff
• Doctor
• Patient
127
Modules of IntelliHealth
• Four Modules
1. Data Acquisition and Preprocessing module
2. Classifier Training and Model Generation
3. Disease Diagnosis Module
4. Report Generation
128
Data Acquisition & Preprocessing Module
Add patient’s detail screen

129
Classifier Training and Model Generation
Load training set and generate model screen

130
Disease Diagnosis Module
Diagnose patient screen displayed to doctor
131
Report Generation
• Report Generation
• Generate medical report for each patient
• Display whether patient shows the symptoms of
disease or not
132
Case Studies
133
Implementing IntelliHealth in Real-time Clinical Practice
Patient showing certain
Entry Clinical examination Suggest medical tests
symptoms of disease
Patient receive test results

Doctor
Test results fed into

Diagnosis by Doctor
proposed DSS
Results
Diagnosis by DSS
comparison
Further investigations by Yes Different

doctor diagnosis
No
Exit Disease diagnosed correctly

134
Case study-1
• Dr Huma Babar, Rawalpindi Institute of Cardiology
• System was trained on 138 patients using 36 attributes
• Tested on 100 patients
Patient_ID 2DEchoResult_Part1 BP-MA_mmHg-uppLim
Age 2DEchoResult_Part2 BP-MA_mmHg-lowLim
Gender 2DEchoResult_Disease AffectedArea1
Protocol 1 AffectedArea2
BMI 2DEchoResult_Disease AffectedArea3
Known_Disease1 2 LV_Myocardium
Known_Disease2 P_Complain1 Defect_Size
Known_Disease3 P_Complain2 Defected_AreaSize
FstMI_Type P_Complain3 Defect_Segment
Angiography_Result1 RestingECGResult1 Via/Non-via
2DEchoResult HeartRate-BI_BPM IsDefected
2DEchoResult_Part HeartRate-MA_BPM I_LVEF
BP-BI_mmHg-uppLim
BP-BI_mmHg-lowLim
• Accuracy of 92.78% and 91.30% F-Measure
135
Case Study-2
• Dr. Lubna Naseem, Pakistan Institute of Medical
Sciences (PIMS), Islamabad
• System was trained on 495 patient’s data consisting 11
attributes
• Tested on 150 patients
Tests Unit Reference value
T.L.C *1000/µL 4.5-11.5
Red cell count million/µL 3.5-5.5
Haemoglobin g/dL 12-15
PCV/HCT Fl 35-55
MCV Fl 75-100
MCH Pg 25.0-35.0
MCHC g/dL 31.0-38.0
Platelet count *1000/µL 100-400
RDW-CV % 11.6-15.0
Neutrophils % 60-70
Lymphocytes % 30-40
• Accuracy of 95.19% and 93.46% F-Measure

136
Contributions Summary
 The research contributes both to the research (HM-
BagMoov) and practice (IntelliHealth Application)
 Proposed a novel combination of classifiers using
proposed enhanced bagging ensemble approach for
multi-layer framework with high accuracy
 Empirical assessment against existing state of the art
over benchmark datasets and evaluation over real time
datasets
 Proposed an IntelliHealth application based on HM-
BagMoov which has significant positive impact on the
use of medical decision support system
137
Conclusion
138
Conclusion
 Proposed multiple ensemble frameworks for disease
diagnosis
 Final ensemble framework adopted layered approach with
enhanced bagging in order to attain maximum accuracy
 Evaluation performed on multiple benchmark and real time
medical datasets to show the performance
 Consistent performance for all medical datasets
 Proposed IntelliHealth application based on HM-BagMoov for
disease diagnosis
 3 Case studies on real time medical datasets in order to prove
the superiority of results
139
Future Work
140
Future Work
 The proposed system can be extended to predict the levels
and types of particular disease
 The next step to this research is a recommender system
where recommendation of medicines and medical treatments
is done for predicted disease
 The proposed system can be applied on other fields data
where classification is required such as Banking, finance and
marketing etc
141
References
1. Thenmozhi,K.,Deepika, P.: Heart Disease Prediction Using Classification with Different Decision Tree Techniques. International
Journal of Engineering Research and General Science Volume 2, Issue 6, October-November, 2015
2. Bashir, S., Qamar, U., & Younus Javed, M. (2014, November). An ensemble based decision support framework for intelligent
heart disease diagnosis. In Information Society (i-Society), 2014 International Conference on (pp. 259-264). IEEE.
3. Chitra, R., Seenivasagam, D.V.: Heart Disease Prediction System Using Supervised Learning Classifier. In: International Journal
of Software Engineering and Soft Computing, Vol. 3, No. 1, March (2013)
4. Shouman,M.,Turner, T., Stocker,R.: Integrating Clustering with Different Data Mining Techniques in the Diagnosis of
Heart Disease. In: Journal of computer science and engineering, Vol 20, Issue 1, (2013).
5. Ghumbre, S., Patil, C., Ghatol, A.: Heart Disease Diagnosis using Support Vector Machine. In: International Conference on
Computer Science and Information Technology (ICCSIT') Pattaya (2011).
6. Shouman, M., Turner, T., Stocker, R.: Using Decision Tree for Diagnosing Heart Disease Patients, In: Proceedings of the 9th
Australasian Data Mining Conference, Ballarat, Australia (2011).
7. Özçift, A. Random forests ensemble classifier trained with data resampling strategy to improve cardiac arrhythmia
diagnosis. Computers in Biology and Medicine, 41(5), 265-271. (2011).
8. Chaurasia, V., Pal, S.: A Novel Approach for Breast Cancer Detection using Data Mining Techniques. International Journal of
Innovative Research in Computer and Communication Engineering. Vol. 2, Issue 1, January 2014
9. K,A. A., Aljahdali, S., Hussain, S.N.: Comparative Prediction Performance with Support Vector Machine and Random Forest
Classification Techniques. International Journal of Computer Applications (0975 – 8887) Vol 69– No.11, (2013).
10. Salama, G.I., Abdelhalim, M.B., Zeid, M.A.: Breast Cancer Diagnosis on Three Different Datasets Using Multi-Classifiers.
International Journal of Computer and Information Technology (2277 – 0764) Vol 01– Issue 01, (2012).
11. Lavanya, D., Rani, K.U.: Ensemble decision tree classifier for breast cancer data. In: International Journal of Information
Technology Convergence and Services (IJITCS) Vol.2. No. 1 (2012).
12. Luo, S. T., & Cheng, B. W. (2012). Diagnosing breast masses in digital mammography using feature selection and ensemble
methods. Journal of medical systems, 36(2), 569-577.
13. Aruna, S., Rajagopalan, S.P., Nandakishore, L.V.: Knowledge based analysis of various statistical tools in detecting breast cancer.
CCSEA, CS & IT, Vol 02, pp. 37–45 (2011).
14. Christobel, A., Sivaprakasam, Y.: An Empirical Comparison of Data Mining Classification Methods. International Journal of
Computer Information Systems,Vol. 3, No. 2, (2011).
15. Lavanya, D., Rani,K.U.: Analysis of feature selection with classification: Breast cancer datasets. Indian Journal of Computer
Science and Engineering (IJCSE), Vol 2, No 5 (2011).
142
References
16. Han, L., Luo, S., Yu, J., Pan, L., & Chen, S. (2015). Rule extraction from support vector machines using ensemble learning
approach: an application for diagnosis of diabetes. Biomedical and Health Informatics, IEEE Journal of, 19(2), 728-734.
17. Gandhi, K.K., Prajapati, N.B.: Diabetes prediction using feature selection and classification. International Journal of Advance
Engineering and Research Development (2014)
18. Stahl, F., Johansson, R., Renard, E.: Ensemble Glucose Prediction in Insulin-Dependent Diabetes. Data driven modeling for
diabetes. Springer (2014)
19. Aslam, M.W., Zhu, Z., Nandi, A.K.: Feature generation using genetic programming with comparative partner selection for diabetes
classification.In: Expert Systems with Applications. Elsevier (2013)
20. NirmalaDevi, M., Appavu, S., Swathi, U.V.: An amalgam KNN to predict diabetes mellitus. In: International conference on
Emerging Trends in Computing, Communication and Nanotechnology (ICE-CCN), (2013)
21. Christobel. Y. A., SivaPrakasam, P.: The Negative Impact of Missing Value Imputation in Classification of Diabetes Dataset and
Solution for Improvement. IOSR Journal of Computer Engineering (IOSRJCE). Volume 7, Issue 4 (2012), PP 16-23
22. Zolfaghari, R.: Diagnosis of Diabetes in Female Population of Pima Indian Heritage with Ensemble of BP Neural Network and
SVM. IJCEM International Journal of Computational Engineering & Management, Vol. 15 (2012)
23. Lee, C.: A Fuzzy Expert System for Diabetes Decision Support Application. Systems, Man, and Cybernetics, Part B: Cybernetics,
IEEE Transactions on (Volume:41 , Issue: 1 ) PP(139-153) 2011
24. Ozcift, A., & Gulten, A. (2011). Classifier ensemble construction with rotation forest to improve medical diagnosis performance of
machine learning algorithms. Computer methods and programs in biomedicine, 104(3), 443-451.
25. Verma, B., & Hassan, S. Z. (2011). Hybrid ensemble approach for classification. Applied Intelligence, 34(2), 258-278.
26. Vijayarani, S., Dhayanand, S., Liver Disease Prediction using SVM and Naïve Bayes Algorithms. International Journal of Science,
Engineering and Technology Research (IJSETR) Volume 4, Issue 4, April 2015
27. Seker, S. E., Unal, Y., Erdem, Z., & Kocer, H. E. (2014). Ensembled correlation between liver analysis outputs. arXiv preprint
arXiv:1401.6597
28. Pahareeya, J., Vohra, R., Makhijani, J., Patsariya, S.: Liver Patient Classification using Intelligence Techniques. International
Journal of Advanced Research in Computer Science and Software Engineering. Volume 4, Issue 2, February 2014
29. Jin, H., Kim, S., Kim, J.: Decision Factors on Effective Liver Patient Data Prediction. International Journal of BioScience and
BioTechnology Vol.6, No.4 (2014), pp.167-178
30. Sugawara, K., Nakayama, N., Mochida, S.: Acute liver failure in Japan: definition, classification, and prediction of the outcome. J
Gastroenterol. 2012 Aug; 47(8): 849–861
143
References
31. Kumar, Y., Sahoo, G.: Prediction of different types of liver diseases using rule based classification model. Technology and
healthcare, vol 21 no. 5, pp 417-432 (2013)
32. Ramana, B.V., Babu, M.P.: Liver Classification Using Modified Rotation Forest. International Journal of Engineering Research and
Development. Volume 1, Issue 6. PP.17-24. (2012)
33. Ramana, B.V., Babu, M.P., Venkateswarlu, N.B.: A Critical Study of Selected Classification Algorithms for Liver Disease
Diagnosis. International Journal of Database Management Systems ( IJDMS ), Vol.3, No.2, (2011)
34. Karthik, S., Priyadarishini, A., Anuradha, J., Tripathy, B.K.: Classification and Rule Extraction using Rough Set for Diagnosis of
Liver Disease and its Types. Advances in Applied Science Research, 2011, 2 (3): 334-345
35. Krawczyk, B., Woźniak, M., & Cyganek, B. (2014). Clustering-based ensembles for one-class classification. Information
Sciences, 264, 182-195.
36. Pushpalatha, S., Pandya, J.: Data model comparison for Hepatitis diagnosis. International Journal of Emerging Research in
Management &Technology. Volume-3, Issue-7. Pp 138-141. July 2014
37. El Houby, E.M.F.: Analysis of Associative Classification for Prediction of HCV Response to Treatment. International Journal of
Computer Applications (0975 – 8887) Volume 63– No.15, February 2013
38. Karthikeyan, T., Thangaraju, P.: Analysis of Classification Algorithms Applied to Hepatitis Patients. International Journal of
Computer Applications (0975 – 8887) Volume 62– No.15, January (2013)
39. Kaya,Y., Murat Uyar, A hybrid decision support system based on rough set and extreme learning machine for diagnosis of
hepatitis disease. 2013 Elsevier, 3429–3438
40. Kumar.M, V., Sharathi.V, V., Devi.B.R.G.: Hepatitis Prediction Model based on Data Mining Algorithm and Optimal Feature
Selection to Improve Predictive Accuracy. International Journal of Computer Applications (0975 – 8887) Volume 51– No.19,
August 2012
41. Lin, S., Chen, S., Chou, S., Enhancing the classification accuracy by scatter-search-based ensemble approach. Applied Soft
Computing. Volume 11, Issue 1, January 2011, Pages 1021–1028
42. Eldin, A.M.S.A.G.: A Data Mining Approach for the Prediction of Hepatitis C Virus protease Cleavage Sites. (IJACSA) International
Journal of Advanced Computer Science and Applications, Vol. 2, No. 12, December 2011
43. Javad Salimi Sartakht, J. S. (2011). Hepatitis disease diagnosis using a novel hybrid method. Elsevier, 570-579.
44. Oh, S., Lee, M., Zhang, B.: Ensemble Learning with Active Example Selection for Imbalanced Biomedical Data Classification.
IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB). Volume 8 Issue 2, March 2011 .Pages 316-325
45. Pattekari, S.A., Parveen, A.: Prediction system for heart disease using Naïve Bayes. In: International Journal of Advanced
Computer and Mathematical Sciences .Vol 3, Issue 3, pp 290-294 , (2012).
144
References
46. Shouman,M.,Turner, T., Stocker,R.: Integrating Naive Bayes and K-means clustering with different initial centroid selection methods in the
diagnosis of heart disease patients. In: Computer science and information technology, pp. 125–137 (2012)
47. Peter, J., Somasundaram.: Probabilistic Classification for Prediction of Heart Disease. Australian Journal of Basic and Applied Sciences,
9(7), Pp 639-643 (2015)
48. K,A. A., Aljahdali, S., Hussain, S.N.: Comparative Prediction Performance with Support Vector Machine and Random Forest Classification
Techniques. International Journal of Computer Applications, Vol 69– No.11, (2013)
49. Kousarrizi, M.R.N., Seiti, F., Teshnehlab, M.: An Experimental Comparative Study on Thyroid Disease Diagnosis Based on Feature Subset
Selection and classification. International Journal of Electrical & Computer Sciences IJECS-IJENS Vol: 12 No: 01 (2012)
50. Chen, J. H., Podchiyska, T., & Altman, R. B., OrderRex: Clinical order decision support and outcome predictions by data-mining electronic
medical records. Journal of the American Medical Informatics Association (2015)
51. Rajkumar, A. and G.S. Reena, Diagnosis Of Heart Disease Using Data mining Algorithm. Global Journal of Computer Science and
Technology, 2010. Vol. 10 (Issue 10).
52. Polat , K., S. Sahan, and S. Gunes, Automatic detection of heart disease using an artificial immune recognition system (AIRS) with fuzzy
resource allocation mechanism and k-nn (nearest neighbour) based weighting preprocessing. Expert Systems with Applications 2007. 32 p.
625–631
53. Kurt,I., Ture, M., Kurum, A.T.: Comparing performances of logistic regression, classification and regression tree, and neural networks for
predicting coronary artery disease. Expert Systems with Applications Volume 34, Issue 1, January 2008, Pages 366–374
54. Liao,J.G., Chin,K.: Logistic regression for disease classification using microarray data: model selection in a large pand small n case.
Bioinformatics. Vol. 23 no. 15 2007, pages 1945–1951
55. Ma,Y., Zuo,L., Chen,J., Luo,Q., Yu,X., Li,Y., Xu,J., Huang,S.,Wang,L., Huang, W., Wang,M., Xu,G., Wang,H.: Modified Glomerular Filtration
Rate Estimating Equation for Chinese Patients with Chronic Kidney Disease. Journal of American society of nephrology. Oct 2006
56. Dubberke, E.R., Reske, K., Olsen, M., McDonald,L., Fraser,V.: Short- and Long-Term Attributable Costs of Clostridium difficile–Associated
Disease in Nonsurgical Inpatients. Clinical Infectious Diseases 2008; 46:497–504
57. KNAUS, WILLIAM A. MD; DRAPER, ELIZABETH A. MS; WAGNER, DOUGLAS P. PhD; ZIMMERMAN, JACK E. MD. APACHE II: A severity
of disease classification system. Critical care medicine. Oct 1985. Vol 3 issue 10
58. Wilson,P.W.F., D’Agostino,R.B., Levy,D., Belanger,A.M., Silbershatz,H., Kannel,W.B.: Prediction of Coronary Heart Disease Using Risk
Factor Categories. American heart association. 1998;1837-1847
59. Ster, B., Dobnikar, A.: Neural networks in medical diagnosis: Comparison with other methods. In: Proceedings of the international conference
on engineering applications of neural networks pp. 427–430. (1996).
60. Georgiou-Karistianis ,N., Gray,M.A., Domínguez D,J.F., Dymowski,A.R., Bohanna,I., Johnston,L.A., Churchyard,A., Chua,P., Stout,J.C.
Egan,G.F.: Automated differentiation of pre-diagnosis Huntington's disease from healthy control individuals based on quadratic discriminant
analysis of the basal ganglia: The IMAGE-HD study. Neurobiology of Disease Volume 51, March 2013, Pages 82–92
145
References
61. Zhang, M.Q. Identification of protein coding regions in the human genome by quadratic discriminant analysis. Proceedings of the national
academy of sciences of the united states of America. Vol 94 no. 2. 2015
62. Maroco,J., Silva,D.,Rodrigues,A., Guerreiro,M., Santana,I.,Mendonç,A.: Data mining methods in the prediction of Dementia: A real-data
comparison of the accuracy, sensitivity and specificity of linear discriminant analysis, logistic regression, neural networks, support vector
machines, classification trees and random forests. BMC research notes. 2011, 4:299
63. Drent,M,. Mulder,PG.,Wagenaar,SS., Hoogsteden,HC.,Velzen-Blad,H.,Bosch,JM.: Differences in BAL fluid variables in interstitial lung
diseases evaluated by discriminant analysis. European respiratory journal. June 1993
64. Srivastava,S., Gupta,M., Frigyik,B.: Bayesian Quadratic Discriminant Analysis. Journal of Machine Learning Research 8 (2007) 1277-1305
65. Ani, R., Augustine, A., Akhil, N. C., & Deepa, O. S. (2016). Random Forest Ensemble Classifier to Predict the Coronary Heart Disease Using
Risk Factors. In Proceedings of the International Conference on Soft Computing Systems (pp. 701-710). Springer India
66. Yang, J., Lee, Y., Kang, U.: Comparison of Prediction Models for Coronary Heart Diseases in Depression Patients. International Journal of
Multimedia and Ubiquitous Engineering Vol. 10, No. 3, pp. 257-268 (2015)
67. Chandna, D. (2014). Diagnosis of heart disease using data mining algorithm.Int. J. Comput. Sci. Inf. Technol.(IJCSIT), 5(2), 1678-1680.
68. Kiruthika, C., Rajini, S.N.S.: An Ill-identified Classification to Predict Cardiac Disease Using Data Clustering. International Journal of Data
Mining Techniques and Applications. Volume: 03, Pages: 321-325 (2014)
69. Jabbar, M.A., Chandra, P., Deekshatulu, B.L.: Heart Disease Prediction System using Associative Classification and Genetic Algorithm.
International Conference on Emerging Trends in Electrical, Electronics and Communication Technologies-ICECIT, 2012
70. Das, R., Turkoglu, I., and Sengur, A.: Effective diagnosis of heart disease through neural networks ensembles. In: Expert Systems with
Applications, Elsevier, pp. 7675–7680. (2009).
71. Chen, H., Yang, B., Liu, J., Liu, D.: A support vector machine classifier with rough set-based feature selection for breast cancer diagnosis.
Expert systems with applications. pp 9014-9022. Vol 38 (2011).
72. Chunekar,V.N., Ambulgekar, H.P.: Approach of Neural Network to Diagnose Breast Cancer on three different Data Set. In: International
Conference on Advances in Recent Technologies in Communication and Computing. pp 893-895 (2009)
73. Kahramanli, H., Allahverdi, N.: Design of a hybrid system for the diabetes and heart diseases. Expert systems with applications. Elsevier.
Volume 35, Issues 1–2, July–August 2008, Pages 82–89
74. Tu,M.C., Shin,D., Shin, D.: Effective Diagnosis of Heart Disease through Bagging Approach. 2nd International Conference on Biomedical
Engineering and Informatics. pp 1-4 (2009).
75. Khalilia, M., Chakraborty, S., & Popescu, M. (2011). Predicting disease risks from highly imbalanced data using random forest. BMC medical
informatics and decision making, 11(1), 1.
76. Hsieh, C. H., Lu, R. H., Lee, N. H., Chiu, W. T., Hsu, M. H., & Li, Y. C. J. (2011). Novel solutions for an old disease: diagnosis of acute
appendicitis with random forest, support vector machines, and artificial neural networks.Surgery, 149(1), 87-93.
77. Ramírez, J., Górriz, J. M., Segovia, F., Chaves, R., Salas-Gonzalez, D., López, M., ... & Padilla, P. (2010). Computer aided diagnosis system
for the Alzheimer's disease based on partial least squares and random forest SPECT image classification. Neuroscience letters, 472(2), 99-
103.
146
References
78. Yang, J., Lee, Y., Kang, U.: Comparison of Prediction Models for Coronary Heart Diseases in Depression Patients. International Journal of
Multimedia and Ubiquitous Engineering Vol. 10, No. 3 (2015), pp. 257-268
79. Peter, J., Somasundaram.: Probabilistic Classification for Prediction of Heart Disease. Australian Journal of Basic and Applied Sciences,
9(7), April 2015. Pp 639-643
80. Kiruthika, C., Rajini, S.N.S.: An Ill-identified Classification to Predict Cardiac Disease Using Data Clustering. International Journal of Data
Mining Techniques and Applications. Volume: 03, June 2014, Pages: 321-325
81. Sørensen, K.P., Thomassen, M., Tan, Q., Bak, M., Cold, S., Burton, M., Larsen, M.J., Kruse, T.A.: Long non-coding RNA expression profiles
predict metastasis in lymph node-negative breast cancer independently of traditional prognostic markers.B reast Cancer Research(2015)
82. Zand. H.K.K.: A comparitive survey on data mining techniques for breast cancer diagnosis and prediction. Indian Journal of Fundamental and
Applied Life Sciences. 2015 Vol.5 (S1), pp. 4330-4339
83. Chaurasia, V., Pal, S.: Data Mining Techniques: To Predict and Resolve Breast Cancer Survivability. International Journal of Computer
Science and Mobile Computing. Vol.3 Issue.1, January 2014, pg. 10-22
84. Chaurasia, V., Pal, S.: A Novel Approach for Breast Cancer Detection using Data Mining Techniques. International Journal of Innovative
Research in Computer and Communication Engineering. Vol. 2, Issue 1, January 2014
85. K,A. A., Aljahdali, S., Hussain, S.N.: Comparative Prediction Performance with Support Vector Machine and Random Forest Classification
Techniques. International Journal of Computer Applications (0975 – 8887) Vol 69– No.11, (2013).
86. Kandhasamy, J.P., Balamurali, S.: Performance Analysis of Classifier Models to Predict Diabetes Mellitus. Procedia Computer Science 47 (
2015 ) 45 – 51
87. Bashir, S., Qamar, U., Khan, F.H.: An efficient rule based classification of diabetes using ID3, C4.5 and CART ensemble. Frontier Information
Technology. IEEE. Islamabad, Pakistan (2015)
88. Gandhi, K.K., Prajapati, N.B.: Diabetes prediction using feature selection and classification. International Journal of Advance Engineering and
Research Development (2014)
89. Tapak, L., Mahjub, H., Hamidi, O., Poorolajal, J., Real-Data Comparison of Data Mining Methods in Prediction of Diabetes in Iran. Healthcare
Information Research. 2013 Sep; 19(3): 177–185.
90. Karthikeyani, V., Begum, I.P., Tajudin, K., Begam, I.S.: Comparative of Data Mining Classification Algorithm (CDMCA) in Diabetes Disease
Prediction. International Journal of Computer Applications (0975 – 8887) Volume 60– No.12, December 2012
91. Julia, A., Vohra, R., Rani, P.: Liver patient classification using intelligent techniques. International Journal of Computer Science and
Information Technologies, Vol. 5 (4) , 2015, 5110-511
92. Sug, H.: Improving the Prediction Accuracy of Liver Disorder Disease with Oversampling. Applied Mathematics in Electrical and Computer
Engineering. Pp 331-335. 2012
93. Houby, E.M.F.: A Framework for Prediction of Response to HCV Therapy Using Different Data Mining Techniques. Advances in
Bioinformatics Volume 2014 (2015)
94. Neshat, M., Sargolzaei, M., Toosi, A.N., Masoumi, A.: Hepatitis Disease Diagnosis Using Hybrid Case Based Reasoning and Particle Swarm
Optimization. Artificial Intelligence. Volume 2012 (2012), Article ID 609718, 6 pages
147
Questions?
148
Thank You!
149
Appendix
150
Top 5 Classifiers for heart disease diagnosis
Classification techniques for heart disease diagnosis
Classifier Avg. Accuracy
Naïve Bayes 76.69%
Decision tree (IG) 75.57%
Decision tree (GI) 75.06%
KNN 69.35%
SVM 79.34%
151
Top 5 Classifiers for multiple diseases diagnosis
Classification techniques for multiple diseases diagnosis
Naïve Bayes 75.44%
Linear Regression 78.32%
Linear Discriminant 76.19%

Analysis
152
Selection of Classifiers [2/2]
Classification techniques for multiple diseases diagnosis
Artificial Neural Network 82.58%

Ensemble
Random Forest 83.97%
153
Combination in BagMoov
 Precision: exactness: what % of tuples that the classifier labeled as

positive (cancer) are actually positive (cancer)
True Positive
Precision= (PPR)
True Positive+False Positive
 Recall (Sensitivity): Completeness: Proportion of negative tuples

that are correctly identified
True Positive
Recall=
True Positive+False Negative
(TPR)
154
Proposed Bagging approach with multi-
objective weighted voting
Training Set
1 2 3 4 ... m
Sampling with
Replacement
Bootstrap Sets
Set 1 1 2 3 ... n
Set 2 1 2 3 ... n
. m and n are number of
. samples, h’s are
.
trained classifiers and
Set t 1 2 3 ... n
p’s are predictions by
the classifiers
h1 h2 h3 ... ht Hypothesis
p1 p2 p3 ... pt Prediction
Multi-Objective Optimized
Weighter Voting
P Final Prediction
155
Limitations
 Identify multiple level of disease using different
classifiers such as svm and LR etc
156
BagMoov
Classifier Accuracy Precision Recall F-Measure
BagMoov 88.83% 89.01% 89.17% 89.09%
BagMoov+ 86.95% 84.33% 85.87% 85.09%

2 classifiers
Poly regression and Perceptron
157

In Tell I Health

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

In Tell I Health

Uploaded by

Copyright:

Available Formats

IntelliHealth: An intelligent medical decision support

system using a novel multi-layer classifier ensemble

• High accuracy is of prime importance in medical field for

• Reduce the margin of diagnostic errors

• Solution: Medical data mining

• Learn from the past to become better in the future

 Decision trees etc

• SOM (Self Organizing Map),

• Predict class label of previously unseen records

• Trade-offs between issues such as accuracy, training

• “No Free Lunch” Theorem

• There is a need of an ensemble model that can combine

• Some of these state of the art techniques are given here

• Open data standard is the norm of Medical Sciences,

1 Heart disease Cleveland heart disease http://archive.ics.uci.edu/ml/datase

K-Mean based on Maximal Frequent Itemset

Bashir,S., et al., [2] 2014 Majority Voting Ensemble 82%

Cascaded Neural Network 85%

Gain ratio Decision Tree 79.1%

Shouman, M., et al., [4] 2013 Naïve Bayes 83.5%

K Nearest Neighbor 83.2%

Support Vector Machine Ensemble 85.05%

Ozcift, A., [7] 2011 Random Forest 89%

Chaurasia, V. et al. [8] 2015 Sequential Minimal Optimization 96.2%

Support Vector Machine Ensemble 75.00%

SMO+J48+NB+KNN (Majority voting) 96.28%

Salama, G.I., et al. [10] 2012 SMO+J48+MLP+KNN (Majority voting) 97%

Lavanya, D., et al. [11] 2012 Decision Tree Ensemble 95.96%

Luo, S., er al. [12] 2012 MultiBoosting Decision Tree 87.9%

S. Aruna, et al. [13] 2011 SVM-RBF Kernel 96.84%

Han, L., et al. [16] 2015 Support Vector Machine 91.2%

Sliding Window Bayesian Model

Genetic programming with

NirmalaDevi, M. et al. [20] 2013 K Nearest Neighbor 92.4%

K Nearest Neighbor with Imputation

Zolfaghari, R. [22] 2012 BP Neural Network and SVM 88%

Lee, C. [23] 2011 Fuzzy Diabetes Ontology 75%

Ozcift, A. et al. [24] 2011 Random Forest 74.47%

Verma, B. et al. [25] 2011 Neural Network Ensemble 80%

Naïve Bayes 70.6%

Seker, S.E. et al. [27] 2014 Bagging 72%

Pahareeya, J. et al. [28] 2014 Random forest 72.4%

Jin, H. et al. [29] 2014 Decision tree 69.4%

Sugawara, K. et al. [30] 2013 Self-Organizing Map 71%

SVM ensemble 72.5%

K Nearest Neighbor 57.97%

Support Vector Machine 62.6%

Karthik, S. et al. [34] 2011 Naïve Bayes 55%

Multilayer Perceptron 70.5%

RBF network 61.5%

Krawczyk, B. et al. [35] 2014 SVM ensemble 85%

Pushpalatha, S. et al. [36] 2014

El Houby, E.M.F. [37] 2013

Karthikeyan, T. et al. [38] 2013

Kaya, Y. et al. [39] 2013

Kumar.M, V. et al. [40] 2012

Lin, S. et al. [41] 2011

Eldin, A.M.S.A.G. [42] 2011

Javad, S.S. [43] 2011

Oh, S., et al. [44] 2011

• Different algorithms perform differently depending on

• None of them appears to be globally superior over the