An Empirical Evaluation of AI Deep Explainable

Tools
Yoseph Hailemariam∗ , Abbas Yazdinejad† , Reza M. Parizi∗ , Gautam Srivastava‡ , Ali Dehghantanha†
∗ College of Computing and Software Engineering, Kennesaw State University, GA, USA
yhailema@students.kennesaw.edu, rparizi1@kennesaw.edu
† Cyber Science Lab, School of Computer Science, University of Guelph, Ontario, Canada
2020 IEEE Globecom Workshops (GC Wkshps) | 978-1-7281-7307-8/20/$31.00 ©2020 IEEE | DOI: 10.1109/GCWkshps50303.2020.9367541

abbas@cybersciencelab.org, adehghan@uoguelph.ca
‡ Department of Mathematics and Computer Science, Brandon University, Manitoba, Canada

srivastavag@brandonu.ca

Abstract—Success in machine learning has led to a wealth of
Artificial Intelligence (AI) systems. A great deal of attention is
currently being set on the development of advanced Machine
Learning (ML)-based solutions for a variety of automated pre-
dictions and classification tasks in a wide array of industries.
However, such automated applications may introduce bias in
results, making it risky to use these ML models in security-
and privacy-sensitive domains. The prediction should be accurate
and models have to be interpretable/explainable to understand
how they work. In this research, we conduct an empirical
evaluation of two major explainer/interpretable methods called
LIME and SHAP on two datasets using deep learning models,
including Artificial Neural Network (ANN) and Convolutional
Neural Network (CNN). The results demonstrated that SHAP
performs slightly better than LIME in terms of Identity, Stability,
and Separability from two different datasets (Breast Cancer
Wisconsin (Diagnostic) and NIH Chest X-Ray) that we used.
Index Terms—AI, Explainable AI, Interpretable Techniques,
ANN, CNN, XAI
I. I NTRODUCTION
In recent years, Artificial intelligence (AI) [1] has been
considered one of the world-transforming research fields. AI
is being utilized in different application domains such as
data security [2], financial trading, advertising, marketing,
healthcare, and blockchain [3]–[5]. AI helps solve very com-
plex problems where it would not have been possible using
traditional algorithmic or other methods. Machine learning
(ML) [6], and particularly deep learning, as the enabling arm
of AI, has gained lots of traction due to its capability in
building intelligent systems in recent years [7], [8].
Designing and building an AI system requires four steps:
Identifying the problem, preparing data, choosing an algo-
rithm, and training the algorithm using the data. About 80
percent of data scientists’ time requires cleaning, moving,
checking, organizing data before even actually using or writing
a single algorithm. So, it is important that the data collected
have to be properly prepared. Once we have identified the
problem and prepared the data, we can select the algorithm.
Knowing the distinction of the algorithms is important to have
a good prediction.
Despite great use and demand for the AI systems and
cerns around the lack of transparency in the underlying
decision-making process. Some application domains even are
reluctant to use AI systems. The reason being the degree
of criticality involved in security-sensitive domains where
prediction should be accurate; otherwise, it would cost a
fatal situation. Security sensitive domains are health-related
domains, monetary domains, mission-critical infrastructures,
etc. The majority of the previous research focuses on the
performance of ML models, when it comes to prediction, not
focusing precisely on explaining the rationale for making the
prediction. Lately, the community has seen the emergence of a
new field, commonly referred to as Explainable AI (XAI) [9]
that provides apparatuses and systems to assist developers and
researchers with creating interpretable and comprehensive ML
models and deploy them with confidence. As a result, there are
interpretability techniques that would help interpret models to
better understand how everything works. Even though there
is a trade-off between performance and interpretability where
the more the model is interpretable such as linear models and
decision trees, the lower the performance would be [10]. It is
still important that a complex model should be interpretable in
a security-sensitive domain. Recently, the European parliament
in May 2018 constitutes a law where it is mandatory for
companies to ‘explain’ any decision taken using machine
learning and deep learning [11]: ”a right of explanation for
all individuals to obtain meaningful explanations of the logic
involved”.
As explained by Doshi-Velez et al. [12], in the context of
ML systems, interpretability is defined as the ability to explain
or to present results in understandable terms to a human. There
is still a question regarding what constitutes a good explana-
tion, where one type of explanation is good for one audience or
bad for the other. In this research, we focus on interpretability
when it comes to safety, which discusses robustness and
security. More specifically, we conduct an empirical evaluation
of two model-agnostic explainers/interpretable methods called
SHAP1 and LIME2 (they are called model-agnostic because
they can explain any machine learning models). We evaluate
1 https://github.com/slundberg/shap

particularly ML-powered applications, there are some con- 2 https://github.com/marcotcr/lime

978-1-7281-7307-8/20/$31.00 ©2020 IEEE

Authorized licensed use limited to: Carleton University. Downloaded on May 29,2021 at 08:34:41 UTC from IEEE Xplore. Restrictions apply.
the explainers’ performance in a security-sensitive domain
using two major deep learning models (Artificial Neural Net-
work (ANN) and Convolutional Neural Network (CNN) ) from
two different datasets (Breast Cancer Wisconsin (Diagnostic)
and NIH Chest X-Ray). With the increasing concerns about
biases in deep classifiers, these types of evaluation will play
an important part in the next generation of networks once
implements (5G and 6G and beyond) to help ensure that the
intelligent deep-based systems are transparent and bias-free.
Anytime future mobile networks need to cope with the growth
in ML, AI, automation, and low latency applications, we will
need such empirical evidence in place beforehand for more
trusted networks [13].
The rest of this paper is structured as follows. We present
the related works in Section II. In Section III, we provide the
methodology used to perform the empirical analysis, including
datasets, metrics, models, and experimental design. Section
IV presents the results and discusses the main observations.
Finally, in Section V, we conclude the paper.
II. R ELATED W ORK
There is a vast number of researches being done in XAI
space with the effort to advance the filed. XAI can be classified
by different criteria such as transparency in ML models (simu-
latability, decomposability, and algorithmic transparency) and
posthoc explainability targets models for ML (text, local and
visual explanations, explanations by example, explanations by
simplification, specific/agnostic XAI techniques [14], [15]).
At the same time, there are a growing number of research
efforts into producing works to evaluate and compare formally
different XAI tools. It is vital to generate empirical knowledge
on how to measure interpretability in different tools and bring
the community to a consensus moving forward.
Doshi-Velez et al. [12] propose three main evalua-
tion approaches for interpretability: application-grounded,
human-grounded, and functionally-grounded. This classifica-
tion ranges from the involvement of humans to no human
involvement. Ideally, it would be better to have a functionally-
grounded evaluation where it would reduce cost and make
it easier to compare results in different domains. Robnik-
Sikonja et al. [16] have proposed some of the properties for
the explanation when doing functionally-grounded evaluation:
Accuracy, Fidelity, Consistency, Stability, Comprehensibility,
Certainty, Importance, Novelty, and Representatives. However,
it is unknown how to assess the properties, and it is not
clear how useful these proprieties for XAI. Miller [17] has
also conducted a comprehensive survey regarding explaining
in Human-Friendly Explanation: Constructiveness, Selectivity,
Social, Focus on the abnormal, Truthful, Consistency, and
General. He states this characteristic makes up a good human
understandable explanation.
Kacper et al. [18] have categorized and systematically
assessed explainable systems along five key dimensions: func-
tional, operational, usability, safety, and validation. In their
work, they compare interpretability methods to better under-
stand their differences.
Authorized licensed use limited to: Carleton University. Downloaded on May 29,2021 at 08:34:41 UTC from IEEE Xplore. Restrictions apply.
With more healthcare-related focus, the authors in [19]
evaluated three agnostic interpretability techniques (LIME,
SHAP, and Anchors) on various healthcare data types. Their
work focuses on various sides of the comparison, such as
identity, stability, separability, similarity, execution time, and
bias detection. Finally, they achieved that LIME has the lowest
performance in the identity metric, but it has the highest
performance in terms of separability in compared to SHAP
and Anchors at all datasets. Moreover, in [20], Adadi et al.
investigated explainable AI for healthcare issues. Indeed, they
tried to clear the black box feature of AI towards interpretable
models. They showed not only recent researches and investiga-
tions regarding the explainability and interpretability of AI but
also consider the ability of XAI on healthcare and medicine
issues. In another work [21], the authors introduced an ap-
proach called Doctor XAI. It is a model-agnostic explainability
approach capable of dealing with sequential, multi-labeled,
ontology data. In this work, the authors offered to explain
Doctor AI that was a multilabel classifier that takes as data
the clinical records of a patient to prognosticate the next visit.
III. M ETHODOLOGY
This section gives the research materials used in the ex-
periment including the dataset, models, explainers and lastly
the approach to our experiment design. The selection of the
research material is based on the observation of the current
practice and picking the recommended tools and frameworks.
A. Datasets
To experiment, we have used two different datasets from the
medical domain which are security (safety) in nature. Security
sensitive domains are more susceptible to adversarial attacks
when it comes to ML models. The first dataset is called Diag-
nostic [22] and its about the detection of breast cancer using
a digitized image of a fine needle aspirate (FNA) of a breast
mass. The dataset classifiers breast cancer detection as benign
or malignant. The dataset has about ten features to constitute
the prediction of benign or malignant. The second dataset
focuses on the NIH Chest X-Ray Dataset. The dataset [23]
is composed of 112,120 X-ray images that clinically diagnose
30,805 unique patients with different labels like Atelectasis,
Pneumonia, Pneumothorax, etc. The reading and diagnosing
chest x-ray images require careful observation and knowledge
of anatomical principles, physiology, and pathology, prone to
adversarial attacks if deep learning models are being used.
B. Deep Models
Deep learning is part of ML in artificial intelligence (AI)
that uses neural networks to learn unsupervised or supervised
from data. Such models have been the popular choice for im-
plementing ML solutions in many security-sensitive domains.
For this experiment, we also favor using deep neural models
because compared to other ML algorithms, as again, they are
known for solving complex problems with effective learning
capability and the ability to extract features on their own.
 Fig. 1. Experiment procedure for Diagnostic dataset with ANN
Fig. 2. Experiment procedure for NIH dataset with CNN
Most importantly, they are very complex to interpret and it is
more advanced technology compared to other AI technology,
thus the explainability of deep learning models is of high
importance. The deep neural networks that we used are CNNs
and ANNs.
1) Artificial Neural Networks (ANNs): ANNs are inspired
by the way the biological nervous system, such as the brain,
process information functions to process information. ANN is
also called Feedforward Neural Network because inputs are
processed only in the forward direction. It is composed of
neurons and consists of input and output layers, as well as (in
most cases) a hidden layer consisting of units that transform
the input into something that the output the layer can use.
2) Convolutional Neural Networks (CNNs): It is notewor-
thy that CNN has been emanated in the 1950s during bio-
logical experiments. David Hubel and Torston Wiesel offered
two types of cells (simple and complex) applied at pattern
recognition [24]. Usually, CNNs are applied in computer
vision applications as well as image processing. However, it is
not just used for those applications and tasks [24]. The main
goal of CNN is to get appropriate trait illustrations of the
input data. CNN is like ANN but what makes it different is
it uses backpropagation to a method to train neural networks
by ”backpropagating” the error from the output layer to the
input layer (including hidden layers). The thing that needs to
be highlighted here is that CNNs require the sum of training
time; nonetheless, they can be computationally acceptable
for AI applications. Furthermore, CNNs are excellent feature
extractors while they may be hard to perform due to the
amount of required math in contrast to other Neural Networks
[24].
Authorized licensed use limited to: Carleton University. Downloaded on May 29,2021 at 08:34:41 UTC from IEEE Xplore. Restrictions apply.
C. Explainers
We used two explainers for this experiment, namely LIME
and SHAP. We have picked these two explainers for being
model agnostic and most commonly used in interpreting deep
models in both industry and academia.
1) LIME: LIME stands for Local Interpretable Model
Agnostic Explanations. The LIME method [15] has been
presented as a local explainability method that focuses on
the supposition that the boundary of a complex ML is linear
near the sample to be interpretable. To understand which
one of the characteristics of a sample takes part in the
prediction, it perturbs the characteristic in its vicinity as well
as observes should there is any treatment later in the model’s
predictions. It interprets the sample of concern by fitting an
explainable model on a perturbed instance near the concerned
input sample. Especially, LIME creates a perturbed instance
near the sample to be interpreted and explained. For instance,
in the perturbed example, LIME reach the prediction from
the example to be interpreted and explained. Indeed, the
perturbed instance will work on the training dataset for the
interpretable model. The method then allocates weights in the
new dataset (training) based on their vicinity to the sample to
be interpreted. Consequently, LIME will fit and interpret and
explain the model on the new dataset (training).
2) SHAP: SHAP applies the game theory approach to con-
sider each specification’s role during the prediction process.
The Shapley amount [25] is a technique from coalitional
game theory. This technique can spread the gain betweens
all players fairly (specifications), where the contribution is
not fair. Especially, the Shapley amount spread the diversity
among the prediction as well as the meddle prediction between
the specification amounts of the sample to be explained. It is
virtually certain that the computation time is one of the most
critical issues in the Shapely amount method. Particularly,
during the precise computation of the Shapley amount, it
should consider all feasible sets of specifications, although
there is no specification of interest. The precise amount of
calculation gets hectic to compute when there are many
specifications since the number of sets will grow exponentially
with the number of specifications. One of the viable ways to
tackle this issue is to apply sampling techniques with a fixed
number of the sample.
D. Measures
The properties to measure used in this paper was inspired by
the Explainability Fact sheets [18]. Specifically, we focused on
safety (security) property to systematically assess the explain-
able system. Sokol et al. [18] proposed that when assessing
an explainable system for safety, it is recommended to focus
on four criteria: Information Leakage, Explanation Misuse,
Explanation invariance, and Explanation Quality. The first
two criteria consider how much information an explanation
reveals about the underlying model and its training data. If
the explanation tells sensitive information about the model,
hackers can use it to exploit it. Explanation invariance con-
cerns with measuring the explanation similarity with the same
dataset and different models but different explainers. Lastly,
Explanation quality concerns with evaluating the quality and
correctness of an explanation underlying models and dataset.
The explainer should only explain the underlying model oth-
erwise, it would be ambiguous. Diogo et al. [26] proposed that
explanation quality can be measured by correctness. The goal
of correctness is to show trust and explain what the model
is doing, without being ambiguous. Explanation invariance
is measured using separability, stability, and identity. The
goal of identity is to show identical instances should have an
identical explanation. If an interpretable technique is run the
same instance several times, it is expected to show the same
explanation otherwise, this shows how unreliable the explainer
is. Stability is somewhat similar to identity, but it also accounts
for a similar instance (instance with a small difference) should
have similar explanations. And lastly, Separability concerns
making sure different instances have a different explanation.
It cannot have identical explanations.
E. Experimental Design and Execution
We carried out the experiment using online machine learn-
ing running services on Kaggle and Google Colab. We used
Kaggle on the image dataset and CNN. And we used Google
Colab for the tabular dataset and ANN. The code for the whole
experimental package for both datasets are hosted in these
GitHub pages3 ,4 .
Fig. 1 shows the experimental design for the first dataset
and the ANN model. In the preparation of the Diagnostic
dataset, we made sure all unnecessary features and raw data
were removed. Then, we applied one-hot encoding to convert
3 https://github.com/yosepppph/BreastCancerANN
4 https://github.com/yosepppph/NIHXrayCNN
Authorized licensed use limited to: Carleton University. Downloaded on May 29,2021 at 08:34:41 UTC from IEEE Xplore. Restrictions apply.
our categorical variables to numerical values, and finally, we
applied feature scaling to normalize the range of independent
variables or features of data. The information that flows
through the network affects the ANN structure because a
neural network changes - or learns, in a sense, based on that
input and output. Thus, we constructed three layers: one entry
layer, one hidden layer, and an output layer. The first two
layers used the ReLU activation function, and the last uses
the Sigmoid function. Fig. 2 shows the experimental design
of NIH Chest Xray and CNN. For the second experiment, we
used the NIH chest Xray dataset. Before we use the dataset,
we performed image preprocessing. We image resizing for
the image preprocessing, changing the color to grayscale, and
generating new images. This dataset’s deep model is CNN
since CNN is known to work well with image datasets. Fig.
3 shows a sample of the actual explanation’s output when we
executed the experimental design for LIME with the ANN
model and Diagnostic dataset, and Fig. 4 shows the same
dataset being explained in SHAP when it is executed.
As can be seen in these figures, the amount of output
value in diagnosis detection at ANN, Diagnostic dataset is
1 and 0.9902 for SHAP and LIME execution explanation
respectively that SHAP has better performance in comparison
with LIME since LIME has 0.0428 loss during diagnosis
detection.
Similarly, Fig. 5 shows a sample of the actual explanation’s
output when we executed the experimental design for LIME
with the CNN model and NIH dataset, and Fig. 6 shows the
same dataset being explained in SHAP when it is executed.
As can be seen in the figures, the SHAP is so exact and
precise to interpret and explain methods on the Xray dataset,
whereas LIME is vague and hard to matches human intuition
for NIH chest Xray dataset.
IV. R ESULTS AND D ISCUSSION
After the experiment execution specified in the previous
section, we collected all the results, which are presented in
this section.
Table I shows the experimental results on the Diagnostic
dataset. The number in the table represents the percentage
of instances that satisfy the defined security metrics. Table II
shows the experimental results on the NIH chest Xray dataset.
TABLE I
E VALUATION OF I NTERPRETABILITY METHODS ON D IAGNOSTIC DATASET
AND ANN
Metrics LIME SHAP
Identity 100% 100%
Stability 94.7% 100%
Separability 100% 100%
In order to measure explanation invariance, as provided in
the measure section, it is through three metrics: Identity, Sep-
arability, and Stability. In Table I, both LIME and SHAP did
well on the majority of the metrics on the Diagnostic dataset
where the only difference is in Stability. SHAP performed
100% and LIME slightly performed lower with 94.7% (it only
 Fig. 3. LIME Execution Explanation (ANN, Breast Cancer Dataset)

Fig. 4. SHAP Execution Explanation (ANN, Breast Cancer Dataset)

Fig. 5. LIME Execution Explanation (CNN, NIH Dataset)

TABLE II A. Limitations
E VALUATION OF I NTERPRETABILITY METHODS ON NIH C HEST XRAY
DATASET AND CNN For this experiment, we couldn’t find meaningful metrics to
use to estimate the last two criteria Information Leakage and
Metrics LIME SHAP
Identity 51% 100%
Explanation Misuse since they both are subjective to find a
Stability 0% 0% threshold that constitutes whether the information is sensitive
Separability 100% 100% or not. We believe more research should be performed to
define quantifiable/objective metrics for information leakage
and explanation misuse. Besides, we weren’t also able to
got wrong 3 samples out of 57 samples). Based on Table I, measure explanation correctness for the image dataset because
we can conclude that SHAP performs better than LIME. In it needs expert validation to assert if an explanation is valid
Table II, both LIME and SHAP didn’t perform as expected or not. This was one of the challenges that we faced during
on stability metrics. One possible reason is since the image’s the conduct of this research.
pixels account for the number of features, having that amount
V. C ONCLUSIONS AND F UTURE W ORK
of pixels makes it harder to group similar samples. The slight
difference between the two explainers was on identity where With the popularity of ML applications and incredibly deep
SHAP performed 100 and LIME performed 54. Just based on learning, many companies and organizations have created
that factor, SHAP performs better than LIME. intelligent systems for many aspects of modern lives. Despite

Authorized licensed use limited to: Carleton University. Downloaded on May 29,2021 at 08:34:41 UTC from IEEE Xplore. Restrictions apply.

Fig. 6. SHAP Execution Explanation (CNN, NIH Dataset)
the advantages such ML-based systems bring about, they are
subjected to bias and being black-boxed in their internal com-
plex working process, which is not widely understood in most
industries. Explainable AI (XAI) is a relatively new research
field that works towards making ML models more transparent
by creating techniques to enable developers/ adopters to be
more confident when they deploy them. As a result, there
have been some tools to put into practice these techniques.
While there is research on this topic, sufficient progress has not
been made regarding understanding such tools’ performance
on a deeper, technical level. This paper experimented with
evaluating two major explainers, LIME and SHAP, for deep
learning models on image and non-image datasets. The results
from this research suggest that SHAP can perform better than
LIME in a security-sensitive domain for both tabular and
image datasets. Future research should consider the potential
effects of empirical approaches more thoroughly in XAI. We
intend to continue our effort to assess the performance of
various explainable and interpretability frameworks on other
medical image datasets like COVID-19. Furthermore, it would
be viable to use other local interpretability tools like layer-wise
relevance propagation (LRP) or Anchors and then compare
such results with our study.
R EFERENCES
[1] S. Makridakis, “The forthcoming artificial intelligence (ai) revolution:
Its impact on society and firms,” Futures, vol. 90, pp. 46 – 60, 2017.
[2] M. Kantarcioglu and F. Shaon, “Securing big data in the age of ai,” in
2019 First IEEE International Conference on Trust, Privacy and Security
in Intelligent Systems and Applications (TPS-ISA), 2019, pp. 218–220.
[3] A. Ekramifard, H. Amintoosi, A. H. Seno, A. Dehghantanha, and R. M.
Parizi, A Systematic Literature Review of Integration of Blockchain and
Artificial Intelligence. Springer International Publishing, 2020, pp. 147–
160.
[4] A. Yazdinejad, G. Srivastava, R. M. Parizi, A. Dehghantanha, K.-
K. R. Choo, and M. Aledhari, “Decentralized authentication of dis-
tributed patients in hospital networks using blockchain,” IEEE Journal
of Biomedical and Health Informatics, 2020.
Authorized licensed use limited to: Carleton University. Downloaded on May 29,2021 at 08:34:41 UTC from IEEE Xplore. Restrictions apply.
[5] D. Połap, G. Srivastava, A. Jolfaei, and R. M. Parizi, “Blockchain tech-
nology and neural networks for the internet of medical things,” in IEEE
INFOCOM 2020 - IEEE Conference on Computer Communications
Workshops (INFOCOM WKSHPS), 2020, pp. 1–6.
[6] P. Louridas and C. Ebert, “Machine learning,” IEEE Software, vol. 33,
no. 5, pp. 110–115, 2016.
[7] M. Saharkhizan, A. Azmoodeh, A. Dehghantanha, K. R. Choo, and
R. M. Parizi, “An ensemble of deep recurrent neural networks for
detecting iot cyber attacks using network traffic,” IEEE Internet of
Things Journal, pp. 1–1, 2020.
[8] A. Yazdinejad, H. HaddadPajouh, A. Dehghantanha, R. M. Parizi,
G. Srivastava, and M.-Y. Chen, “Cryptocurrency malware hunting: A
deep recurrent neural network approach,” Applied Soft Computing,
vol. 96, p. 106630, 2020.
[9] D. Gunning, “Explainable artificial intelligence (xai),” Defense Ad-
vanced Research Projects Agency (DARPA), nd Web, vol. 2, p. 2, 2017.
[10] R. El Shawi, Y. Sherif, M. Al-Mallah, and S. Sakr, “Interpretability in
healthcare a comparative study of local machine learning interpretability
techniques,” in 2019 IEEE 32nd International Symposium on Computer-
Based Medical Systems (CBMS), 2019, pp. 275–280.
[11] R. Guidotti, A. Monreale, S. Ruggieri, F. Turini, F. Giannotti, and
D. Pedreschi, “A survey of methods for explaining black box models,”
ACM Computing Surveys, vol. 51, no. 5, 2018.
[12] F. Doshi-Velez and B. Kim, “Towards a rigorous science of interpretable
machine learning,” arXiv preprint arXiv:1702.08608, 2017.
[13] A. Yazdinejad, R. M. Parizi, A. Dehghantanha, H. Karimipour, G. Sri-
vastava, and M. Aledhari, “Enabling drones in the internet of things
with decentralized blockchain-based security,” IEEE Internet of Things
Journal, pp. 1–1, 2020.
[14] A. B. Arrieta], N. Dı́az-Rodrı́guez, J. D. Ser], A. Bennetot, S. Tabik,
A. Barbado, S. Garcia, S. Gil-Lopez, D. Molina, R. Benjamins,
R. Chatila, and F. Herrera, “Explainable artificial intelligence (xai):
Concepts, taxonomies, opportunities and challenges toward responsible
ai,” Information Fusion, vol. 58, pp. 82 – 115, 2020.
[15] M. T. Ribeiro, S. Singh, and C. Guestrin, ““why should i trust you?”:
Explaining the predictions of any classifier,” in Proceedings of the 22nd
ACM SIGKDD International Conference on Knowledge Discovery and
Data Mining, ser. KDD ’16, 2016, p. 1135–1144.
[16] M. Robnik-Šikonja and M. Bohanec, Perturbation-Based Explanations
of Prediction Models. Springer International Publishing, 2018, pp.
159–175.
[17] T. Miller, “Explanation in artificial intelligence: Insights from the social
sciences,” Artificial Intelligence, vol. 267, pp. 1–38, 2019.
[18] K. Sokol and P. Flach, “Explainability fact sheets: a framework for
systematic assessment of explainable approaches,” in Proceedings of the
2020 Conference on Fairness, Accountability, and Transparency, 2020,
pp. 56–67.
[19] R. El Shawi, Y. Sherif, M. Al-Mallah, and S. Sakr, “Interpretability in
healthcare a comparative study of local machine learning interpretability
techniques,” in 2019 IEEE 32nd International Symposium on Computer-
Based Medical Systems (CBMS), 2019, pp. 275–280.
[20] A. Adadi and M. Berrada, “Explainable ai for healthcare: From black
box to interpretable models,” in Embedded Systems and Artificial Intel-
ligence, V. Bhateja, S. C. Satapathy, and H. Satori, Eds. Singapore:
Springer Singapore, 2020, pp. 327–337.
[21] C. Panigutti, A. Perotti, and D. Pedreschi, “Doctor xai: an ontology-
based approach to black-box sequential data classification explanations,”
in Proceedings of the 2020 Conference on Fairness, Accountability, and
Transparency, 2020, pp. 629–639.
[22] “Breast cancer wisconsin (diagnostic) data set.” [Online]. Available:
https://www.kaggle.com/uciml/breast-cancer-wisconsin-data
[23] “Nih chest x-rays data set.” [Online]. Available: https://www.kaggle.
com/nih-chest-xrays/data
[24] A. Khan, A. Sohail, U. Zahoora, and A. S. Qureshi, “A survey of the
recent architectures of deep convolutional neural networks,” Artificial
Intelligence Review, pp. 1–62, 2019.
[25] D. Butnariu and T. Kroupa, “Shapley mappings and the cumulative
value for n-person games with fuzzy coalitions,” European Journal of
Operational Research, vol. 186, no. 1, pp. 288–299, 2008.
[26] D. V. Carvalho, E. M. Pereira, and J. S. Cardoso, “Machine learning
interpretability: A survey on methods and metrics,” Electronics, vol. 8,
no. 8, p. 832, 2019.