IEEE/CAA JOURNAL OF AUTOMATICA SINICA, VOL. 10, NO.
4, APRIL 2023
Explainable, Domain-Adaptive, and Federated
Artificial Intelligence in Medicine
Ahmad Chaddad, Qizong Lu, Jiali Li, Yousef Katib, Reem Kateb, Camel Tanougast,
Ahmed Bouridane, Senior Member, IEEE, and Ahmed Abdulkadir
   Abstract—Artificial intelligence (AI) continues to transform
data analysis in many domains. Progress in each domain is driven
by a growing body of annotated data, increased computational
resources, and technological innovations. In medicine, the sensi-
tivity of the data, the complexity of the tasks, the potentially high
stakes, and a requirement of accountability give rise to a particu-
lar set of challenges. In this review, we focus on three key
methodological approaches that address some of the particular
challenges in AI-driven medical decision making. 1) Explainable
AI aims to produce a human-interpretable justification for each
output. Such models increase confidence if the results appear
plausible and match the clinicians expectations. However, the
absence of a plausible explanation does not imply an inaccurate
model. Especially in highly non-linear, complex models that are
tuned to maximize accuracy, such interpretable representations
only reflect a small portion of the justification. 2) Domain adapta-
tion and transfer learning enable AI models to be trained and
applied across multiple domains. For example, a classification
task based on images acquired on different acquisition hardware.
3) Federated learning enables learning large-scale models without
exposing sensitive personal health information. Unlike centralized
AI learning, where the centralized learning machine has access to
the entire training data, the federated learning process iteratively
updates models across multiple sites by exchanging only parame-
ter updates, not personal health data. This narrative review cov-
Manuscript received July 29, 2022; revised October 28, 2022; accepted
November 10, 2022. This work was supported in part by the National Natural
Science Foundation of China (82260360) and the Foreign Young Talent
Program (QN2021033002L). Recommended by Associate Editor Jiayi Ma.
(Corresponding author: Ahmad Chaddad.)
Citation: A. Chaddad, Q. Z. Lu, J. L. Li, Y. Katib, R. Kateb, C. Tanougast,
A. Bouridane, and A. Abdulkadir, “Explainable, domain-adaptive, and
federated artificial intelligence in medicine,” IEEE/CAA J. Autom. Sinica, vol.
10, no. 4, pp. 859–876, Apr. 2023.
A. Chaddad, Q. Z. Lu, and J. L. Li are with the School of Artificial
intelligence, Guilin Universiy of Electronic Technology, Guilin 541004,
China. A. Chaddad is also with The Laboratory for Imagery, Vision and
Artificial Intelligence, Ecole de Technologie Superieure, Montreal, Quebec
H3C 1K3, Canada (e-mail: ahmadchaddad@guet.edu.cn; qizong.lu@hotmail.
com; indigo.aomg@gmail.com).
Y. Katib is with the College of Medicine, Taibah University, Madinah
42353, Saudi Arabia (e-mail: yousef_katib@hotmail.com).
R. Kateb is with College of Computer Science and Engineering, Taibah
University, Madinah 42353, Saudi Arabia (e-mail: rkatb@taibahu.edu.sa).
C. Tanougast is with the Laboratory of Design, Optimization and Modeling
Systems, University of Lorraine, Metz 57070, France (e-mail: camel.tanougast
@univ-lorraine.fr).
A. Bouridane is with the Center for Data Analytics and Cybersecurity
(CDAC), University of Sharjah, Sharjah 27272, United Arab Emirates (e-
mail: abouridane@sharjah.ac.ae).
A. Abdulakadir is with the Laboratoire de recherche en neuroimagerie,
Centre Hospitalier Universitaire Vaudois, Mont Paisible 16, 1011, Lausanne,
Switzerland (e-mail: Ahmed.Abdulkadir@pennmedicine.upenn.edu).
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/JAS.2023.123123
859
ers the basic concepts, highlights relevant corner-stone and state-
of-the-art research in the field, and discusses perspectives.
    Index Terms—Domain adaptation, explainable artificial intelli-
gence, federated learning.
  
I.  Introduction
RTIFICIAL intelligence (AI)–understood here as the
A capability of computers to transform input data to elicit
an appropriate response [1], has a huge potential to transform
many aspects of our lives. The methodological advances that
drive the progress generally focus on core issues like solving
specific tasks more accurately [2], creating models that gener-
alize to unseen data [3], or addressing fundamentally new
tasks [4]. Within these innovation cycles, the application of
medical data faces particular challenges. Personal health
related information is sensitive and therefore not readily
sharable. When it is shared, the data itself is highly complex
and heterogeneous. There is not only a large variation between
individuals, but there is also measurement noise and system-
atic effects due to acquisition instruments and protocols. This
noise, uncertainty, and heterogeneity in the data and labels
create distinctly challenging sets of problems. The goal of AI
in medicine is to support the clinical decision process. The
decisions in clinical processes can have grave consequences,
sometimes observed with a substantial delay and usually
uncorrectable. Furthermore, state-of-the-art AI models tend to
have a large number of parameters, which are necessary to
learn complex interaction and generalize well. Thus, even
when being deterministic, AI models do not behave intu-
itively and may fail or behave unexpectedly [5]. This percep-
tion of the black-box behavior leads to subjective mistrust and
objectively limited reliability of an unknown extent.
With the growing availability of extensive medical data and
the increase in computational power, AI has become more
important in clinical practice for early disease detection, accu-
rate diagnosis, predictions, and prognosis [6]. However, AI
models–mostly instantiated as deep artificial neural networks,
are sufficiently complicated to be non-intuitive. Thus, the
computer models are perceived as a black boxes [7]. Then, the
black-box nature leads to mistrust of predictive models by
clinicians [8]. For this reason, explainable artificial intelli-
gence (explainable AI) was proposed in order to make AI
models more transparent and easier to understand and inter-
pret [9]. Explainable AI can thus complement human percep-
tion of the naked data [10], [11] and will contribute to making
AI models in medicine more trusted.
Computer-aided diagnosis is increasingly involved in clini-
cal decision processes to advance the goal of personalized
 860
medicine [12]. However, some challenges limit its application
in clinical practice, such as domain transfer [13]. Specifically,
it assumes that the training and test sets are independent and
identically distributed (IID) using machine learning (ML)
models. In reality, the distributions of the training and test sets
may not be identical. This leads to domain shift when we
transfer the model [14]. Despite advanced deep learning mod-
els [15], the error increases proportionately with different dis-
tributions between the training data set and the test data set
[13]. Therefore, how to solve the domain transfer problem is a
key to applying AI to clinical tasks. Domain adaptation (DA)
breaks the traditional assumption that the distribution of the
training and test data sets must be consistent with ML [16].
The data set is divided into source and target data in this con-
text. The source data is related to the task to be solved, and the
target data (with no labels or only a few labels) is related to
the task. Note that there are always domain shifts between two
domains [16]. In medical imaging, there are significant differ-
ences within data sets of the same type/modality due to vari-
ous equipment, sites, protocols, etc [17]. Domain adaptation
techniques aim to reduce these differences.
More and more attention has been paid to data privacy in
recent years [18]. In clinical topics, personalized models
require protection of personal health information [19]. Feder-
ated learning (FL) can integrate data from private sources
without sharing private data itself [20]. Specifically, FL pro-
tects users’ privacy primarily by exchanging non-personal
data such as updated models between agents (clients) of the
learning federation [21].
This paper aims to provide a brief narrative review of
explainable/interpretable, domain-adaptive, and federated AI
for clinical applications. Fig. 1 illustrates a yearly growing
number of publications in the field. These search results indi-
cate a clear trend. However, it is worth noting that the search
technique is not 100 percent sensitive because synonymous
terms may be used for similar techniques; for instance, trans-
fer learning is a type of DA. In addition, some research works
may discuss one of the topics superficially yet being counted.
This review provides a brief overview of XAI, DA and FL
and categorizes techniques within each field. For each topic,
we motivate its use in medical research and healthcare appli-
cations and discuss challenges and potentials for feature
research.
The remainder of this paper is structured as follows. Sec-
tion II consists of the fundamental concepts for the three
methodological approaches. We discuss in Section III the
applications of XAI, DA and FL in medical research and
healthcare. Finally, Section IV concludes by providing per-
spectives of challenges and untapped potential.
II.  Methodological Foundation and Background
A. Explainable Artificial Intelligence
Human-centric medical decision processes are designed to
be trustworthy because of the high stakes that are involved.
Isolated outputs from AI models derived from deterministic,
yet highly non-linear networks that consist of artificial neu-
rons organized in many–up to a thousand and more, layers
IEEE/CAA JOURNAL OF AUTOMATICA SINICA, VOL. 10, NO. 4, APRIL 2023
100
Topic
Search results
XAI
FL
50 DA
0
2016 2017 2018 2019 2020 2021 2022
Year
Fig. 1. Number of publications indexed on PubMed (https://pubmed.ncbi.
nlm.nih.gov) that matched the search queries related to the topics in this sur-
vey. FL: \federated learning” AND ((medicine) OR (healthcare)), XAI:
((\explainable AI”) OR (\explainable artificial intelligence”)) AND
((medicine) (healthcare)), DA: (\domain adaptation”) AND ((medicine) OR
(healthcare)). The number for the year 2022 is a projection based on the
assumption that publications appear at the same rate for the remaining two
months of the year 2022.
[22] leave no room for interpretation and provide no explana-
tion of the prediction. AI models are therefore casually con-
sidered as black boxes because the “How?” and “Why?” a
machine arrives at its conclusion is in general not understand-
able by humans [9]. The field has a rich and nuanced taxon-
omy that is not always used consistently. Explainability and
interpretability–sometimes used interchangeably, relate to
human-focused understanding [23]–[25]. A machine is under-
standable if its decisions are based on descriptive and relevant
information that is recognizable by humans. In this context,
descriptive means how well the interpretation method cap-
tures the learned relationships and relevant means how useful
the interpretation is [26]. Explainable AI models strive for
high causability–the extent to which a human can casually
understand the decision making process of artificial intelli-
gence . Explainability and related research fields are elements
of building reliable and trusted AI systems [10].
XAI elements are either injected into the model architecture
or are external. The former leads to models that intrinsically
increase transparency of the model, whereas the latter intro-
duces explainability post-hoc [27]. Transparency is achieved
by highlighting relevance of features, simplification, and
visual data that is relevant and meaningful for the practition-
ers [28].
B. Domain-Adaptive Artificial Intelligence
Generalizability is the implicit ability to use a trained pre-
dictive model from one or more source domains in a different,
but related, target domain. In contrast, domain adaptation is
achieved through explicit modelling of domain differences.
Assume we train a classifier model using data from a source
domain D s for some task T s, and to learn the same or similar
task Tt using target data Dt . If the two domains have the same
distribution (D s ∼ Dt ) and the tasks are identical, then the
model trained for task T s on D s can also be used for the task
Tt in Dt . If the two domains are not from the same distribu-
tion (D s / Dt ), then the models trained on the source domain
D s are likely to perform worse on Dt . DA is utilized to exploit
similarities of tasks and learn differences in data domains to
produce models for task Tt in Dt [29]. Fig. 2 shows an exam-
ple of homogeneous domain adaptation, this is when the same
CHADDAD et al.: EXPLAINABLE, DOMAIN-ADAPTIVE, AND FEDERATED ARTIFICIAL INTELLIGENCE IN MEDICINE
Source
data
Domain
adaptation
Target
data
Source domain: Target domain:
Separating hyperplane in source domain:
Fig. 2. Illustration of homogeneous domain adaptation in which the source
and target distributions consist of the same features but have different distri-
butions. Red and blue colors represent the source and target data, respectively.
Circles and squares represent class labels. Domain adaptation aims to apply-
ing models (dashed line) to data from a target domain (blue) that were learned
using data from a source domain (red). In the example shown, the domain
adaptation consists of shifting and rotating the data points such that after the
transformation, the source, and target domains overlap. The separating classi-
fication hyperplane, shown as grey dashed line, only accurately classifies the
data points from the target data after domain adaptation.
task is applied to the same features that have different distri-
butions. Other types of DA methods are outlined in the taxon-
omy below.
1) Supervised, Semi-Supervised, and Unsupervised Domain
Adaptation: Depending on the availability of annotation, DA
applications can be classified as supervised [30], semisuper-
vised [31], and unsupervised [32]. In supervised DA models,
labels of all training instances are available. On the other
hand, in semi-supervised learning, only a part of the training
data is annotated. Learning with no labels is considered unsu-
pervised. Specifically, most studies focus on unsupervised DA
models due to insufficient numbers of labeled samples. For
example, a novel unsupervised DA algorithm proposes to
eliminate the domain differences for reducing the label noise
[17]. While, a deep domain adaptation method is proposed to
transfer domain knowledge from a well-labeled source
domain to a partially labeled target domain [33]. This method
minimizes the differences in the domain that have an impor-
tant clinical impact for disease diagnosis.
2) Single- and Cross-Modality: Medical image derived from
many types of scanner (e.g., magnetic resonance imaging, CT,
X-ray) that are related to texture differences between the
source and target domains [34]. This phenomenon is called the
modality difference. When based on acquisition, DA methods
are categorized as either single- or cross-modality DA. In sin-
gle-modality DA, the adaptations happen within a single type
of data [35]. In clinical practice, medical images are acquired
on hardware from different vendors and at different centers
[36]. Because different modalities provide complementary
information and vary in acquisition cost and expertise, a diag-
nosis can be established with different modalities. Cross-
modal DA helps to make models applicable across modalities.
For example, some models can be transferred between CT and
MRI images [37]. In addition, bidirectional cross-modality
DA between MRI and CT images has been applied using the
synergistic image and feature alignment framework (SIFA)
861
[38]. Despite the practicability of DA, there are always
remaining risks of inherent biases in the data [39] or unfair-
ness in models [40].
3) Single- and Multi-Source Domain Adaptation: Depend-
ing on the number of source domains, it is possible to classify
DA into single-source DA (SDA), multi-source DA (MDA)
and source-free DA [41]. For example, trained models using
single-source domains lack generalization. Multi-source DA
collects labeled data from multiple sources with different dis-
tributions [42] and implicitly learn to be robust. For example,
MDA learning can alleviate the difficulties caused by align-
ing multimodal characteristics between multiple domains [43].
Although MDA is advanced, MDA applications are challeng-
ing due to the heterogeneity of data between domains. For this
reason, most existing studies are based on SDA [44]. Another
issue could be related to the source domains, cause to data pri-
vacy or lack of storage space for small devices [45]. A gener-
alized source-free domain adaptation (G-SFDA) was pro-
posed to possess source domain features via transfer learning
[46].
4) One- and Multi- Step DA: On the basis of the distance
between the source and the target domain, domain adaptation
methods can be roughly categorized into two types: one-step
and multistep domain adaptation methods. The one-step DA
methods often simply align the feature distributions of the two
domains by minimizing their distances. But it is difficult for
the network to minimize the huge domain discrepancy.
Specifically, in remote domain transfer learning, when the dis-
tance between the source domain and the target domain is too
long [47], it is impossible to achieve and can even affect the
performance of the learning model in the target domain. This
situation is called negative transfer [48]. In order to avoid this
situation, it is necessary to find an “intermediate bridge” to
connect the source and the target domain. That is called multi-
step DA. It can be transformed into one-step domain adapta-
tion by choosing an appropriate intermediate domain. For
example, through efficient instance selection, transfer fea-
tures are successfully applied between remote domains and
effectively solve the remote domain problem [49].
5) Shallow and Deep DA: According to the type of model,
we can divide the DA method into shallow and deep DA. Usu-
ally, shallow DA leverages feature engineering and is used
with traditional machine learning. But few timely reviews the
emerging deep learning based methods. Unlike the shallow
DA, deep DA methods can leverage deep networks to learn
more transferable representations by embedding DA in a deep
learning pipeline [50]. In addition, deep DA integrates a fea-
ture into the learning model to improve performance metrics
[13].
These methods may reduce the differences in the domain
that can have a positive impact in clinical research. Specifi-
cally, more effort is required to fully apply such methods in
clinical topics to avoid bias.
C. Federated Learning
Federated learning (FL) is a way of distributed collabora-
tive machine learning in which the training data is not shared.
This concept of data-private collaborative learning [51], that
862
initially was motivated in the context of mobile devices in
which each one has only a small fraction of the data, equally
applies to medical application in which the data is distributed
across hospitals [52].
This framework works by collaborating multiple devices to
update a machine learning model without sharing their pri-
vate data under the supervision of a central server. An exam-
ple in medicine is the COVID-19 chest scan engine, which
was built by many researchers and medical practitioners from
different parts of the world. In the COVID-19 pandemic,
along with increased privacy concerns, FL is mainly useful to
support the diagnosis and detection of COVID-19. This was
done by coordinating massive hospitals to build a common AI
model [53].
We now give a general formal definition of FL. Let N be the
number of data clients–for example hospitals, {H1 , H2 , . . . ,
HN }, all of whom wish to contribute to the training of a pre-
dictive model by combining their respective data
{D1 , D2 , . . . , DN }. A traditional method is to combine all data
and use D = D1 ∪ D2 , . . . ∪ DN to train a model Mall . FL is a
learning process in which the data clients collaboratively train
a model MFL , in which process any data client Hi does not
share its data Di to others. In addition, the accuracy of MFL ,
denoted as VFL should approach the performance of Mall ,
Vall . Formally, let δ be a non-negative real number, if
|VFl − Vall | < δ. (1)
Then, the federated learning algorithm has an accuracy loss
of δ.
FL is particularly attractive for smart healthcare, by coordi-
nating multiple hospitals to perform an AI training without
sharing raw data [54]. However, FL is an emerging research
area and has not yet gained much trust in the medical commu-
nity; therefore, it has required more research, particularly with
respect to security and privacy aspects. Although this frame-
work sounds ideal in theory, it is not immune to attacks.
Moreover, its current development is not mature enough to be
expected to solve all privacy issues by default. The most spe-
cific security threats facing the FL framework are communica-
tion bottlenecks, poisoning, backdoor attacks, and inference-
based attacks, which can be considered the most critical to FL
privacy [20]. In this context, FL is represented in three main
groups a) privacy protection [55], b) effective communication
[56] and c) data heterogeneity [57].
1) Privacy Protection: For generalization of the predictive
model in medical data analysis, recent work focuses on con-
sidering multisite samples with multidimensionality. How-
ever, data sharing between institutions has been controlled by
laws such as the United States Health Insurance Circulation
and Accountability Act (HIPAA) [58] and the European Gen-
eral Data Protection Regulation (GDPR) [59]). FL demon-
strates its feasibility in sharing and protecting the privacy of
20 medical institutions and centers [60]. FL is represented by
models that are trained simultaneously at each site. These
models are periodically aggregated and redistributed, requir-
ing only the weight transfer of learned model between institu-
tions. This step eliminates the requirement of sharing data
directly [52]. Although FL can efficiently protect the data pri-
IEEE/CAA JOURNAL OF AUTOMATICA SINICA, VOL. 10, NO. 4, APRIL 2023
vacy, their application still bares some security-related risks
[61].
2) Effective Communication: Communication is a critical
bottleneck in federated networks [62]. FL network can consist
of thousands of agents contributing model updates. To obtain
high-performance models, the multi-participant situation is
inevitable. In this context, network communication is slower
than local computation by many orders of magnitude due to
limited resources such as bandwidth, energy, and power. To
solve this problem, two key aspects can be used: a) reducing
the total number of communication rounds and b) reducing the
size of the messages transmitted in each round [63]. It can per-
form client-side filtering, reduce the frequency of updates, and
P2P (peer-to-peer) learning (e.g., compression) [64].
3) Data Heterogeneity: Currently, data samples in hospitals
are not independent and distributed identically (IID). It leads
to an increase in the bias of the model [65]. A proposed initial
shared model used for hospitals that can easily adapt it to the
local dataset [66]. Through intelligently choosing the partici-
pant, it can reduce the times of communication in FL and
counterbalance the bias introduced by IID data. However, the
data are not always IID type. To systematically and intu-
itively understand the current FL method in different data dis-
tributions, new studies have been conducted a comprehensive
experimental comparison between the new algorithms [67].
This means that FL can solve the problem of multisite data,
such as in multiple clinical institutions [68].
Regardless of these advanced FL techniques, more efforts
are needed to consider these models in healthcare field.
III.  Methods
Deep learning models are widely applied in medicine. How-
ever, these models require more interpretability to simplify the
steps of the prediction process. Despite advances in XAI mod-
els, there is a huge need to consider XAI with DA and FL
when the topics are related to medicine to achieve the goal of
personalized medicine. For example, DA-based XAI models
have the ability to improve the performance in computer aided
diagnosis [69]. But the same model will not be able to pro-
vide a feasible performance using multi-institutional data. For
this reason, FL is a good option for data sharing under the
condition of protecting data privacy [70]. However, there is an
additional issue related to intra- and inter- site difference of
distributions. This leads to a domain shift between sites.
Recent work proposes to integrate unsupervised domain adap-
tation (UDA) into the FL framework [71]. For example, UDA
guides the model to learn domain-agnostic features through
adversarial learning or a specific type of batch normalization
[47]. Furthermore, DA techniques such as mixture of experts
(MoE) with adversarial domain alignment can be used to
improve the accuracy of different sites in the FL learning
setup [72]. These techniques will be presented below in three
subsections related to DA, XAI and FL.
A. Current Methods of Explainable Artificial Intelligence
With high-performance deep learning models of increasing
complexity, the way specific results were obtained is not
accessible to humans. A number of methods were proposed to
CHADDAD et al.: EXPLAINABLE, DOMAIN-ADAPTIVE, AND FEDERATED ARTIFICIAL INTELLIGENCE IN MEDICINE 863

TABLE I
Summary of the XAI Models Used for Clinical Applications
Application XAI / Explainability level Prediction model Reference
Preclinical relevance assessment Model combination / G GNN, FNN [89]
Detection and prediction for Alzheimer’s disease SHAP / L & G RF [90]
Deterioration risk prediction of hepatitis SHAP, LIME, PDP / L & G RF [91]
Non-communicable diseases prediction DeepSHAP / L & G DNN [92]
Spinal posture classification LIME / L SVM [93]
Classification of estrogen receptor status SmoothGrad / L DCNN [94]
Differential diagnosis of COVID-19 Grad-CAM / G CNN [95]
Prediction of mortality Shapley additive / L RNN [96]
Medical image segmentation Combination / L & G CNN, Attention Mechanism [97]
Early detection of Parkinson’s disease LIME / L VGG-16 [98]
COVID-19 diagnosis Grad-CAM ++, LRP / L DNN [99]
Evaluation of cancer in Barrett’s esophagus 5 XAI methods / L AlexNet, SqueezeNet, ResNet50, VGG16 [100]
Heart anomaly detection SHAP and Occlusion maps / L CNN, MLP [101]
Distinguish VTE patients DeepLIFT / L ANN [102]
Detection of ductal carcinoma in situ LRP / L Resnet-50 CNN [103]
Lesion prediction Activation Maps, Saliency Maps / L U-net [104]
Early detection of COVID-19 Saliency Maps / L Fuzzy-enhanced CNN [105]
EEG emotion recognition SmoothGrad, saliency maps / L CNN [106]
Cerebral hemorrhage detection and localization Grad-CAM / L ResNet [107]
Macular disease classification t-SNE, model simplification / L CNN [108]
Automated diagnosis and grading of ulcerative colitis Grad-CAM / L CNN [109]
Diagnostic for breast cancer Causal-TabNet / G GNN [110]
Prediction of depressive symptoms LIME / L RNN [111]
Early prediction of antimicrobial multidrug resistance SHAP / L & G LSTM [112]
Predictions of clinical time series CAM / Local FCN [113]
L: Local; G:Global; ANN: Artificial Neural Network; GNN: Graph Neural Networks; FNN: Fuzzy Neural Network; DNN: Deep Neural Networks; RNN:
Recurrent Neural Network; FCN: Fully Convolutional Network; DCNN: Deep Convolutional Neural Network; VGG-16: GG-Very-Deep-16 CNN; VGG-19:
GG-Very-Deep-19 CNN; Fuzzy-enhanced CNN: Fuzzy-enhanced Convolutional Neural Network; RF: Random forest; SVM: Support Vector Machine; MLP:
Muti-Layer Perceptron; LSTM: Long Short-Term Memory network; PDP: Partial Dependence Plot; t-SNE: t-distributed Stochastic Neighbor Embedding.
Seven Network: SqueezeNet, Inception, ResNet, ResNeXt, Xception, ShuffleNet, DenseNet; Five XAI methods: Saliency, guided backpropagation, integrated
gradients, input gradients, and DeepLIFT.

increase explainability in recent years, 17 of which were dis-
cussed in a recent survey [73] and a selection of which is
listed in Table I and discussed in following paragraphs. These
techniques are mainly divided into two categories, i) explain-
ability in modeling and ii) explainability after modeling (see
Fig. 3).
1) Explainability in Modeling: Explainability is derived
from modifications of the model architecture. This process
can be represented in simplification (and/or quantification)
layers and loss function, or by adding an explainable network
and visualization module. For example, CNN architectures
consist of convolutions, pooling, and fully connected layers;
To increase the explainability of CNN, layers could be quanti-
fied, modified, and combined with a visual model. Replace-
ment of convolutional layers with max-pooling layers pro-
vides better visualization without losing performance metrics
[74]. Another scenario related to CNN is represented by
adding attention network and modifying the loss function
[75]. Recent studies have been quantified the convolution lay-
ers by Gaussian mixture model and/or texture features for
clinical classifications [76]–[81]. However, there are only a
few studies focused on explainable hidden layers.
2) Explainability After Modeling: It requires many experi-
ments to balance model accuracy based on post-hoc inter-
pretability (explainability after modeling). It is grouped into
the following categories:
a) Feature relevance analysis: It considers the importance
of features in the prediction model (e.g., layer-wise relevance
back propagation (LRP)). LRP is an explainable AI model that
explores the relation between output and input data by gener-
ating separate correlation maps for each individual image.
LRP with explainability concept is also used to prune CNN. It
finds the most relevant units (i.e., weights or filters) with their
relevance scores [82]. Deep-LIFT is also a method to improve
the effectiveness of feature learning and improve the inter-
pretability of the model itself. It works by comparing the acti-
vation of each neuron to its reference activation and assigns
scores according to the difference between the two [83], [84].
b) Visual explanation: To further improve model visualiza-
tion, heat maps, saliency maps, or class activation methods are
864 IEEE/CAA JOURNAL OF AUTOMATICA SINICA, VOL. 10, NO. 4, APRIL 2023

DATA POST-HOC
ACQUISITION ARCHITECTURE MODIFICATION EXPLAINABILITY

Fully
Convolution connected Dense
Input Pooling Output Depth 1 Depth 2 ……
layers
CNN L
Input S Output LRP
T
CNN M
φi

Clinical image Layer modification Model combination Feature relevance analysis

D
Input G e
Classifier
Discriminative loss
A n F
Soft
Reward
P s C
function e
max
LSTM
Relevance loss
Cross ωk
ω2
Data entropy loss ω1
Similarity
Clinical report Attention network score Loss modification CAM
Input mapping

Fig. 3. Example of flowchart for the explainable artificial intelligence model, 1) data acquisition consists of images and clinical report for each of patients.
2) Example of four methods during modeling: it can modify layer (Layer modification) to simplify the structure or combined (Model combination) with other
model to build a full explainable model. In addition, an attention network could be used for visualization and even to modify the loss function. 3) The two most
used methods after modeling: feature relevance analysis and visual explanation.

considered. Among these functions, class activation mapping
(CAM) is the most widely used. In CNN, it can improve
explainable capabilities by replacing the fully connected layer
with the global average pooling (GAP) layer [85].
However, CAM requires modifying the structure of the
original model by retraining the model. For example, another
version of CAM known as Grad-CAM that is a deep neural
network [86]. It uses the gradient information that is related to
the convolutional layer and decision of interest. Whether it is
CAM or Grad-CAM, the visualization results they generate
are not visually enough. In addition, a Score-CAM model pro-
poses to manage the dependence on gradients by different
ways for obtaining linear weights [87].
3) Explainability Level: The explainability scale is divided
into different categories in different situations. For example,
learning models they are applied to. Specifically, LIME uses
explainability models (such as linear models and decision
trees) to approximate the target predictions from the black-box
model. It detects the relationships between the output of the
black-box model by slightly perturbing the input in the XAI
model. For example, LIME demonstrated feasible explana-
tions in artificial neural networks by altering the input feature
and fitting it to a linear model [114]. The SHAP explains how
each feature affects the predicted value (known as Shaply). It
is computed based on the average of the marginal contribu-
tions of all permutations (permutations of each feature). It pro-
vides different explanations for a particular model (e.g., fea-
ture importance visualization, linear formulation). With SHAP
[115], it illustrates how the design parameters affect perfor-
mance using DNN features.

XAI is divided into five scales according to ethology in

explaining animal intentionality [88]. According to the trans-
parency of the model, XAI is divided into simulatability,
decomposability, and algorithmic transparency [25]. Further-
more, XAI with deep learning models could be grouped into
a) local and b) global explanations. For local explanation, the
model considers a local approximation of how the black box
works. It mainly focuses on explanation of data instances. The
global explanation describes the whole mechanism of the
model. Generally, a group of data instances is needed to gen-
erate one or more explanation maps.
Table I summarizes XAI in deep learning models applied to
clinical tasks. We note that CNNs are the most widely used
deep learning models and gradient-based explainability is the
most applied principle. The level of explainability is mostly
local.
Agnostic model In addition to the previous XAI methods,
shapley additive explanations (SHAP) and local explainable
model-agnostic explanation (LIME) are agnostic to the deep
B. Domain Adaptation With Deep Learning
We present the main steps in DA model by summarizing the
existing literature. We grouped the existing methods into 1)
DA-related data, 2) DA-related features, and 3) building and
adversarial domain adaptation. In addition, we report these
models in Table II. We note that the problem of domain bias
occurs when the target domain is different from the source
domain. This bias type limits the performance of transfer deep
learning model that leads to disable the generalization of
model.
1) Domain Adaptation Related Data: It assumes that DA
can solve the problem of domain shift. However, there is a
bias due to different hospital equipments. Instance Re-weight-
ing Adaptation is a method to solve this issue when the
domain bias is small. It removes samples from the source
domain that mismatch the similar distribution in the target
domain. For example, samples with higher correlation with
target samples are assigned larger weights [116]. The assigned
CHADDAD et al.: EXPLAINABLE, DOMAIN-ADAPTIVE, AND FEDERATED ARTIFICIAL INTELLIGENCE IN MEDICINE 865

TABLE II ity, and transmission efficiency. Therefore, a method that can

Summary of Recent Domain Adaptation Models adapt source-trained models towards target distributions with-
Used For Clinical Topics
out accessing source data will be promising. To adapt a
Cross-
Application Method domain Topics Reference model, samples from the source and target domains can share
Gastric epithelial similar representations to provide similar predictions [122].
GAN N Y [125]
tumor These models need to be pre-trained using the source domain
Multiple sclerosis
lesions CNN N N [122] data and re-trained by the target domain [122], which will
Arrhythmia CNN N N [126]
increase the processing time. Therefore, these types of predic-
heartbeat tive models are not preferred. Another scenario based on
Bio-metric CycleGAN Y S and N [127] adversarial domain adaptation is currently one of the main-
identification
Medical image DDA-Net Y N [128] stream methods. It is known that deep models (e.g., adversar-
Cardiac PnP-AdaNet Y N [129] ial domain adaptation) can use the transformed features for
Brain dementia domain adaptation [123]. For example, an unsupervised
SVM Y Y [130]
identification method based DA used for predicting glaucoma fundus [124].
Brain dementia
identification AD2A N N [131] Specifically, they designed a loss function that learns the
Autism spectrum MSDA/MVSR N Y [132]
source region features and possesses the original labels
disorder unchanged. An improved adversarial domain adaptation net-
Cardiac CycleGAN Y N [133] work has been considered for tumor image diagnosis with few
Lung ADDA N N [134] and/or no labels [125].
Histopathological HisNet-SSDA Y S [135] Table II reports the DA methods used for clinical applica-
Breast cancer DADA N Y [136] tions. We find that the dominant research approaches in recent
Pancreatic cancer GCN N N [137]
years are related to adversarial networks, with a lower domi-
nant for DA methods based on features and data. It observes
Atrial fibrillation 3DU-Net N Y [138]
that the most of these approaches based on unsupervised DA.
Y: Yes; N: No; S: Semi-supervised; GAN: Generative Adversarial Network;
CNN: Convolutional Neural Network; CycleGAN: Cycle-Consistent This is due to the limitation of medical data. These contribu-
Generative Adversarial Networks; DDA-Net: Dual Domain Adaptation tions provide a solid foundation for future DA methods
Network; DASC-Net: Domain Adaptation based Self-Correction model;
PnP-AdaNet: Plug-and-Play Adversarial Domain Adaptation; SVM:Support applied in medicine.
  

Vector Machine; AD2A: Attention-guided Deep Domain Adaptation;

MSDA: Multi-Source Domain Adaptation; MVSR: Multi-View Sparse
Representation; ADDA: Adversarial Discriminative Domain Adaptation;
HisNet-SSDA: a deep transferred semi-supervised domain adaptation;
DADA: Depth-Aware Domain Adaptation; GCN: Graph Convolutional
Networks;3D U-Net: three dimension U-net.
data are then re-entered into the training model to reduce the
problems arising from domain shifts [117]. This idea has been
developed with a task of re-weighting mechanism that consid-
ers loss function with dynamically update weights [118].
Regardless of the advances algorithms and methods used in
DA related data, these models are able to only solve the prob-
lem of small domain bias.
2) Domain Adaptation Related Features: It is based on the
feature similarity between the source and the target domains.
For example, the feature transformation strategy can solve the
bias problem by transferring the source and target samples
from the original feature space to the new feature representa-
tion space [119]. A novel feature-based transfer learning algo-
rithm for remote domains requires only a small portion of
labeled target samples from a different domain [120]. How-
ever, transfer features can only reduce the distribution vari-
ance from a single perspective, and it is a challenge to find an
optimal transfer learning method for a given dataset. For this
reason, the distribution variance was reduced from multiple
perspectives by applying multi-feature-based transfer learning
methods [121].
3) Building an Adversarial Domain Adaptation: Data- and
feature-based DA aims to align a labeled source domain and
an unlabeled target domain, which requires accessing the
source data. It will raise concerns about data privacy, portabil-
C. Federated Learning Modeling
FL updates the models to achieve high performance by con-
sidering co-modeling step between different data sources (or
client). FL is divided into the following categories (see Fig.
4):
1) Horizontal Federate Learning: Horizontal federated
learning (HFL) is a system in which all the parties share the
same feature space. It does the training process as follows: a)
Participants first download the latest model from the server
individually and train the model based on the local data, b)
The trained model with its parameters (i.e., cryptographic gra-
dients) is uploaded to the server, and c) The server aggregates
the gradients of each user and returns the updated model to
each participant to update local models [154]. With this
advantage, HFL is widely used in clinical topics [155].
2) Vertical Federated Learning: Vertical federated learning
(VFL) lets multiple parties that possess different attributes
(e.g., features and/or labels) of the same data entity (e.g., a
person) to jointly train a model. Its learning process consists
of a) encrypted sample alignment and b) an encryption train-
ing model. a) Encrypted sample alignment: Using encryption-
based user ID alignment technology to align common users.
These common users data are then considered for b) encryp-
tion training model. However, in learning process, VFL
requires multiple communications between all participants
that need a high processing time. For this reason, many stud-
ies have developed an asynchronous training algorithm based
on vertically partitioned data [156]. We note that many exist-
ing methods focus on HFL when all feature sets and labels are
available.
866
Encrypted gradients Encrypted gradients
Updates
Updates Encrypted gradients
Updates
Hospital 1 Hospital 2
(a)
Fig. 4. Example of federated learning based on a) horizontally and b) vertically technique; In horizontally federated learning, multiple clients (Hospitals) with
the same data structure collaborate to train a predictive/classifier model with the help from server. In vertically federated learning: First, align the common users
in different participants. Then, it performed the encrypted training model based on these common users.
3) Federated Transfer Learning: Federated transfer learn-
ing (FTL) generalizes the concept of federated learning and
emphasizes that collaborative modeling learning can be per-
formed on any data distribution and entity [157]. For example,
participants first compute and encrypt their intermediate
results with their gradients and losses. The gradients and
losses are then collected and decrypted by a collaborator.
Finally, the participants receive the aggregated results that are
used to update the respective models. We note that FTL uses
homomorphic encryption for security and the polynomial dif-
ference approximation for a) privacy and b) prevention of data
leakage [154]. Unlike HFL that possesses the original data and
models locally, FTL makes the participants encrypt the gradi-
ents before data transmission, and adding random masks to
avoid the participants from getting information of each other’s.
In healthcare applications, FTL can not only provide a stan-
dard framework, but also achieve efficient classifications and
avoid the problem of unbalanced classes [158].
4) Federated Optimization: Compared with traditional mach-
ine learning, FL focuses on significant variability of system
features on each device in the network (systems heterogene-
ity), and non-identically distributed data across the network
(statistical heterogeneity) [159]. Furthermore, aggregation FL
requires defining an aggregation strategy, such as a method to
combine the local models coming from the clients into a
global one. For example, the standard and simplest aggrega-
tion strategy is federated averaging (known as FedAvg [160]).
Another strategy is called FedProx [161] is a generalization of
FedAvg with some modifications to address data and the het-
erogeneity of the system. Furthermore, FedProx is considered
to explore the interactions between statistical and systematic
heterogeneity [159]. In addition, there is a new version of
FedAvg called FedAvg+ based on personalization and cluster
FL [162]. Also, FedEM model is based on the mixed distribu-
tion assumption with the expectation maximum algorithm
[163]. Unfortunately, these optimization algorithms are still
rarely applied in healthcare topics.
Table III reports FL techniques used recently for healthcare
IEEE/CAA JOURNAL OF AUTOMATICA SINICA, VOL. 10, NO. 4, APRIL 2023
Federated model
Encrypted model training
Encrypted entity alignment
Hospital 3 Participant A Participant B
(b)
data. It observed that the accuracy of FL model under the opti-
mization algorithm is close to the central model, which
demonstrates the potential of federated learning. However, the
number of clients studied is still limited to 100 [148]. More-
over, most of the data from these clients are derived from pri-
vate datasets or multi-site datasets, which will limit the devel-
opment of FL in practical medical applications [164]. So far,
FL for healthcare is still in its early stages. The preliminary
investigations provide a good foundation for future work,
regardless of whether they involved replicating FL environ-
ments or doing limited experiments across hospitals [165].
D. Overview of Methods and Datasets Used for XAI, DA, and FL
Table IV compares methods of XAI, DA and FL based on
key strengths and weaknesses and lists code and datasets
within each topic. To overcome the limited availability of
medical datasets, many of the XAI, DA and FL based on deep
learning techniques employ transfer learning from a large
dataset such as ImageNet [166]. Thus, we include the original
datasets and code used for the recent and popular techniques.
For example, Grad-CAM is one of the common XAI methods
used for multitask in medical topics like grading of ulcerative
colitis [109], detecting cerebral hemorrhage [167], and pre-
dicting COVID-19 [67], [168] with a high degree of image
interpretation. LIME also shows a feasible rate in multitask
such as spinal posture classification [93], detection of Parkin-
son’s disease [98], and predicting depressive symptoms [111].
Many other techniques used recently (e.g., Shapley Additive
Explanations, Graph LIME, etc.) are almost based on deep
learning models that need high power and performance com-
puters.
DA in high-dimensional complex data like medical imaging
relies on deep learning to implicitly model nonlinear transfor-
mations [169]. For example, the difference of the feature dis-
tribution between source and target domains is narrowed by
the deep domain confusion model for cross subject recogni-
tion [170]. Recent algorithms like deep CORAL [171], deep
adaptation networks [172], deep subdomain associate adapta-
CHADDAD et al.: EXPLAINABLE, DOMAIN-ADAPTIVE, AND FEDERATED ARTIFICIAL INTELLIGENCE IN MEDICINE 867

TABLE III
Summary of Federate Learning Models Used for Clinical Topics
Application Model / Learning / optimization n Performance (Local / Central / FL models) Reference
Predicting clinical outcomes EXAM / HFL /—— 20 AUC: 0.79/ - / 0.92 [139]
Precision medicine RETAIN / HFL / FedAvg 42 AUC: - / 0.73 / 0.62 [140]
Spinal cord gray matter segmentation DNN / HFL / FedAvg / FedPRox 4 Sensitivity: - / 0.76 / 0.78 [141]
Prostate cancer diagnosis 3D AH-Net / HFL /—— 3 Dice: 0.812–0.872/ - / 0.89 [52]
Detection diabetic retinopathy AlexNet / FTL / FedAvg / FedProx 5 Acc: 0.81-0.92/ - / 0.65-0.90 [142]
Prediction in COVID-19 patients LASSO, MLP / HFL /—— 5 AUC: - / 0.719-0.822 / 0.786-0.836 [143]
Image reconstruction U-net / HFL / FedAvg 4 SSIM: 0.94/ - / 0.93 [144]
Computational pathology CNN / HFL / 3 AUC: - / 0.985±0.004 / 0.976±0.007 [145]
Predicting outcomes in patients ResNet-34 / HFL / FedAvg 20 AUC: 0.79 / - / 0.92 [146]
Multimodal melanoma detection EfficientNet / HFL / FedAvg 5 Acc: - / 0.83 / 0.83 [147]
Arrhythmia detection of Non-IID ECG ——/ HFL / EWC 100 F1-score: - / — / 0.80 [148]
Joint training ——/ HFL / FedACS 3 Acc: - / — / 0.70 [149]
MultiView training CNN / VFL/—— 2 Acc: 0.62/ - / 0.71 [150]
Prognostic prediction ELM / HFL /—— 4 Acc: - / 0.89 / 0.8899 [151]
Designing ECG monitoring healthcare system CNN / FTL /—— 3 Acc: - / 0.94 / 0.92 [70]
Multi-center imaging diagnostics 3D-CNN, ResNet-18 / HFL /—— 4 AUC: - / 0.85 / 0.86 [152]
COVID-19 detection ResNet / HFL / FedMoCo 3 Acc: - / 0.96 / 91.56 [153]
Local model: training alone in client; Central model: use all client data for training; FL model: model under FL training methods; n: number of institutions;
EXAM: Electronic Medical Record (EMR) chest X-ray AI model; HFL: Horizontally Federate Learning; VFL: Vertically Federated Learning; FTL:
Federated Transfer Learning; DNN: Deep Neural Networks; CNN: Convolutional Neural Network; 3D-CNN: three-Dimensional Convolutional Neural
Network; AlexNet, EfficientNetB7, ResNet, ResNet-34, ResNet-18, U-net: a class in CNN; AUC: Area Under Curve; Acc: Accuracy; 3D AH-Net: 3D
Anisotropic Hybrid Network; LASSO: Least Absolute Shrinkage and Selection Operator; MLP: Multilayer Perceptron; Dice: Dice coefficient range; Kaapa
score: is a measure of classification accuracy; SSIM: Structural Similarity Index Measure; PSNR: Peak-Signal-to-Noise Ratio; ELM: Extreme Learning
Machine; RETAIN: has two parallel RNN branches merged at a final logistic layer; EWC: elastic weight consolidation; FedAvg: Federated Averaging;
FedACS: FedAvg with adaptive client sampling; FedMoCo: a robust federated contrastive learning framework; FedProx: A optimal method to tackle
heterogeneity in federated networks.

tion network (DSAAN) [173] and emotional domain adversar-
ial neural network (EDANN) [174] are recommended to
reduce domain differences. Despite the advances in DA mod-
els, few algorithms were applied to medical data [169]. An
example in medical imaging is the explicit harmonization of
uni-modal brain imaging data across multiple sites with vary-
ing acquisition hardware [175].
Recent studies of FL provide promising results in health-
care applications. For example, data from 20 institutes across
the globe to train a model (EXAM: electronic medical record
(EMR) chest X-ray AI model), for predicting COVID-19
[139]. This model is based on HFL that considers differential
privacy to mitigate risk of data “interception” during site-
server communication. In [142], three main models using
standard transfer learning, FedAvg, and Federated Proximal
frameworks, respectively, are considered to diagnose diabetic
retinopathy. In [176], a new study proposes federated disen-
tangled representation learning for unsupervised brain
anomaly detection using MR scans from four different institu-
tions. To solve the problems of security and trustworthiness,
densely connected CNN based on FedAvg is proposed and
used for COVID-19 [177]. Due to the limitations of sharing
the data across hospitals and countries, more collaborations
will improve the feasibility of FL in medical applications.
IV.  Conclusion and Future Perspective
The rate at which the complexity and volume of digital
patient data are acquired, far exceeds the rate at which expert
clinicians can be trained to analyze it. Therefore, it is neces-
sary to employ artificial intelligence to keep up with the
growth of the data. Deep learning based image analysis
reached the performance of expert clinicians in making refer-
ral recommendations based on widespread available optical
coherence tomography [194]. In many scenarios, however,
artificial intelligence is not yet readily able to replace expert
clinicians because the applicable AI methods are not plug-in
replacements for human decision-making. In neuroimaging,
for example, the transition to the clinics did not take place
despite many successes in research studies [195]. Applica-
tions of AI with a human in the loop relieve the clinician from
some–typically tedious work. Atri-U, for example, is an appli-
cation of AI in the clinical routine in which the AI back-end
proposes a segmentation, two landmarks, and a time point to
compute the volume of the left atrium [196]. The clinician that
bares the responsibility can either accept, modify, or ignore
the proposals. The latter scenario introduces a non-disruptive
innovation into an existing diagnostic process with a substan-
tially reduced average process time without compromising
quality. Further developments of such AI tools for clinical
practice will depend on the key technologies discussed previ-
ously, and for each of which we will conclude with a sum-
mary.
1) Explainable AI: We found that the most XAI with deep
learning addresses post-hoc interpretability to increase face
value, like the CNN-CAM. We also found that the explain-
ability levels are limited local explainability. This means that
868 IEEE/CAA JOURNAL OF AUTOMATICA SINICA, VOL. 10, NO. 4, APRIL 2023

TABLE IV
The Strengthts and Weaknesses of XAI, DA, and FL Techniques
Technique Strengths Weaknesses Code Data Reference
Explainable Artificial Intelligence
- Simple to use - The weight of saliency map depends on the
- No need to change the network model and data set [67], [86],
Gradient-weighted Class structure - Sometimes prominent areas cannot be C1 D1, D2 [109], [167],
Activation Mapping - Efficient to compute correctly labeled [168]
- High degree of image explanation - Poor robustness
Local Explainable Model- - Uses a simple model for local - Poor robustness [93], [98],
C2 D3
agnostic Explanation explanation - Inefficient calculation [111], [178]
- Simple to use - Inefficient calculation
Graph LIME *C3 D4 [91], [179]
- Trustworthy - instable
Shapley Additive - Useful for interpretation - Inefficient computation C4 D5, D6 [96], [180]
Explanations - Repeatability
Domain Adaptation
- Only adapts to one layer of network
- Reduce the distribution difference
Deep Domain Confusion between the source and target domains -optimal
A single fixed kernel may not be the C5 D7 [170], [181]
kernel
- Domain difference problem cannot be
Deep Adaptation Networks - Reduce domain differences C5 D7 [182], [183]
solved by semi-supervised learning
Deep Subdomain Adaptation - The weight could be defined flexible - The weight value of network maybe could C5 D8 [184], [185]
Network not be the best
- Solve the dynamic distribution
Dynamic Adversarial adaptation problem in adversarial - Can not handle regression problems C5 D7 [186], [187]
Adaptation Networks networks
Federated Learning
- Can use large, and heterogeneous - Bias may occur due to limitations in data
EXAM / HFL data-sets C6 D9 [139]
quality.
- Robust and generalizable
AlexNet / FTL / FedAvg & - Sub-optimal security
- Prediction rate is high − D10–D12 [142]
FedProx - Data accuracy pitfalls
- Mitigate the statistical heterogeneity
- Good generalizability with clinical
CNN / HFL / FedDis - Privacy concerns C7 D13–D17 [176]
applications
- Flexibility
- High performance metrics - Weak federated training process for the
3D-CNN / HFL / FedAvg - Provided visual explanations unstable internet connection. C8 D18 [177]
- Suitable for transfer learning - Inefficient computation
FedDis: Federated Disentred representation learning for unsupervisived brain operatory segmentation; EXAM: Electronic Medical Record (EMR) chest X-ray
AI model, “−” not available, “*” not official, C: code, D: data.
C1 https://github.com/zhoubolei/CAM, https://github.com/Cloud-CV/Grad-CAM
C2 https://github.com/marcotcr/lime
C3 https://github.com/WilliamCCHuang/GraphLIME
C4 https://github.com/slundberg/shap
C5 https://github.com/jindongwang/transferlearning/tree/master/code/DeepDA/, https://github.com/ting2696/Deep-Symmetric-Adaptation-Network.
C6 https://ngc.nvidia.com/catalog/models/nvidia:med:clara_train_covid19_exam_ehr_xray
C7 https://github.com/albarqounilab/
C8 https://github.com/HUST-EIC-AI-LAB/UCADI
D1 http://host.robots.ox.ac.uk/pascal/VOC/voc2007
D2 ImageNet Object Localization Challenge [166]
D3 https://github.com/marcotcr/lime-experiments
D4 Datasets are from Cora and Pubmed.
D5 [188]
D6 http://yann.lecun.com/exdb/mnist
D7 [189]–[193], ultrasoundcases.info
D8 https://github.com/jindongwang/transferlearning/blob/master/data/dataset.md, [186]
D9 https://www.kaggle.com/datasets/mariaherrerot/eyepacspreprocess
D10 https://www.adcis.net/en/third-party/messidor
D11 https://ieee-dataport.org/open-access/indian-diabetic-retinopathy-image-dataset-idrid
D12 https://www.kaggle.com/datasets/mariaherrerot/aptos2019
D13 https://www.oasis-brains.org
D14 http://adni.loni.usc.edu/data-553samples/access-data/
D15 http://lit.fe.uni-lj.si/tools.php?lang=eng
D16 https://smart-stats-tools.org/lesion-challenge-5562015
D17 https://www.med.upenn.edu/sbia/brats2018/data.html
D18 https://www.nhsx.nhs.uk/covid-19-response/data-and-covid-19/national-covid-19-chest-imaging-database-nccid/

the model understanding can only explain the model output
rather than the internal decision-making. Global explainabil-
ity is desirable in clinical tasks to achieve trust. More specifi-
cally, the practical XAI application is inevitable to consider
people who do not have the relevant AI background. Addi-
tional explanations such as manual instructions will be greatly
recommended.
2) Domain Adaptation: Domain adaptation can effectively
promote the transfer of models between different domains.
However, data heterogeneity and label validity are still a criti-
cal challenge. Many DA methods can eliminate the domain
bias and the difference in data distribution, such as data-based
methods and feature-based methods. Unfortunately, these
methods are too rigid and require a lot of processing time.
CHADDAD et al.: EXPLAINABLE, DOMAIN-ADAPTIVE, AND FEDERATED ARTIFICIAL INTELLIGENCE IN MEDICINE
Unlike adversarial DA, which shows strong performance, it
can effectively reduce domain differences and improve gener-
alization performance.
3) Federated Learning: Federate learning is not well applied
to healthcare data, and most studies focus on horizontal feder-
ated learning, which does not fully exploit the potential of fed-
erated learning. Only few optimization algorithms can achieve
performance similar to the central model. We note that FL
faces the challenges of poor data quality and insufficient
model accuracy. This is due to the absence of standard and
uniform data collection among institutions. Therefore, it men-
tions that the standard unified data with model security protec-
tion mechanism is the key to improving the deep learning
model for clinical tasks.
4) Outlook to Further Challenges: In the past decade, many
methods originating in computer vision and other non-medi-
cal fields found applications in medical research healthcare
with some delay and minor adaptations. If this trend contin-
ues, we expect to see more of the emerging XAI, DA, and FL
methods in medicine for tasks that are similarly structured as
in non-medical fields. We note, however, that healthcare pro-
cesses often incorporate irregularly sampled, longitudinal,
multi-modal data for decision-making. The imputation of
missing data can lead to inaccurate prediction that may result
in biased estimations [197], [198]. With strong assumptions
and simplified models, clinical decision processes can be
modeled and optimized using mixed linear models that natu-
rally handle missing data, as long as the input data is low-
dimensional and in a homogeneous domain [199]. To train
deep learning models with similar flexibility but more com-
plex data, much larger training data sets are required. Com-
bined, FA may contribute to leverage massive pools of pri-
vate data that will be transformed into a common reference
domain with DA to train interpretable and thus more trustwor-
thy AI models using XAI.
References
[1] S. C. Shapiro, Encyclopedia of Artificial Intelligence. 2nd ed. New
York, USA: Wiley, 1992.
[2] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. A. Ma, Z.
H. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L.
Fei-Fei, “ImageNet large scale visual recognition challenge,” Int. J.
Comput. Vis., vol. 115, no. 3, pp. 211–252, Dec. 2015.
[3] H. D. Li, J. C. Li, X. M. Guan, B. H. Liang, Y. T. Lai, and X. L. Luo,
“Research on overfitting of deep learning,” in Proc. 15th Int. Conf.
Computational Intelligence and Security, Macao, China, 2019, pp.
78−81.
[4] O. Vinyals, I. Babuschkin, W. M. Czarnecki, M. Mathieu, A. Dudzik,
J. Chung, D. H. Choi, R. Powell, T. Ewalds, P. Georgiev, J. Oh, D.
Horgan, M. Kroiss, I. Danihelka, A. Huang, L. Sifre, T. Cai, J. P.
Agapiou, M. Jaderberg, A. S. Vezhnevets, R. Leblond, T. Pohlen, V.
Dalibard, D. Budden, Y. Sulsky, J. Molloy, T. L. Paine, C. Gulcehre,
Z. Y. Wang, T. Pfaff, Y. H. Wu, R. Ring, D. Yogatama, D. Wünsch,
K. McKinney, O. Smith, T. Schaul, T. Lillicrap, K. Kavukcuoglu, D.
Hassabis, C. Apps, and D. Silver, “Grandmaster level in StarCraft II
using multi-agent reinforcement learning,” Nature, vol. 575, no. 7782,
pp. 350–354, Nov. 2019.
[5] A. Nguyen, J. Yosinski, and J. Clune, “Deep neural networks are
easily fooled: High confidence predictions for unrecognizable images,”
in Proc. IEEE Conf. Computer Vision and Pattern Recognition,
Boston, USA, 2015, pp. 427−436.
[6] G. Varoquaux and V. Cheplygina, “Machine learning for medical
869
imaging: Methodological failures and recommendations for the
future,” NPJ Digit. Med., vol. 5, no. 1, p. 48, Apr. 2022.
[7] M. Gaur, K. Faldu, and A. Sheth, “Semantics of the black-box: Can
knowledge graphs help make deep learning systems more interpretable
and explainable?” IEEE Internet Comput., vol. 25, no. 1, pp. 51–59,
Jan.-Feb. 2021.
[8] A. Adadi and M. Berrada, “Explainable AI for healthcare: From black
box to interpretable models,” in Embedded Systems and Artificial
Intelligence, V. Bhateja, S. C. Satapathy, and H. Satori, Eds.
Singapore, Singapore: Springer, 2020, pp. 327−337.
[9] G. Yang, Q. H. Ye, and J. Xia, “Unbox the black-box for the medical
explainable AI via multi-modal and multi-centre data fusion: A mini-
review, two showcases and beyond,” Inf. Fusion, vol. 77, pp. 29–52,
Jan. 2022.
[10] A. Holzinger, “The next frontier: AI we can really trust,” in Proc. Joint
European Conf. Machine Learning and Knowledge Discovery in
Databases, 2021, pp. 427−440.
[11] A. Holzinger, M. Dehmer, F. Emmert-Streib, R. Cucchiara, I.
Augenstein, J. Del Ser, W. Samek, I. Jurisica, and N. Díaz-Rodríguez,
“Information fusion as an integrative cross-cutting enabler to achieve
robust, explainable, and trustworthy medical artificial intelligence,”
Inf. Fusion, vol. 79, pp. 263–278, Mar. 2022.
[12] H. P. Chan, L. M. Hadjiiski, and R. K. Samala, “Computer-aided
diagnosis in the era of deep learning,” Med. Phys., vol. 47, no. 5,
pp. e218–e227, Jun. 2020.
[13] H. Guan and M. X. Liu, “Domain adaptation for medical image
analysis: A survey,” IEEE Trans. Biomed. Eng., vol. 69, no. 3,
pp. 1173–1185, Mar. 2022.
[14] G. Bode, S. Thul, M. Baranski, and D. Müller, “Real-world application
of machine-learning-based fault detection trained with experimental
data,” Energy, vol. 198, p. 117323, May 2020.
[15] Y. G. Lei, B. Yang, X. W. Jiang, F. Jia, N. P. Li, and A. K. Nandi,
“Applications of machine learning to machine fault diagnosis: A
review and roadmap,” Mech. Syst. Signal Process., vol. 138, p.
106587, Apr. 2020.
[16] C. C. Zhang and Q. J. Zhao, “Deep discriminative domain adaptation,”
Inf. Sci., vol. 575, pp. 599–610, Oct. 2021.
[17] Y. F. Zhang, Y. Wei, Q. Y. Wu, P. L. Zhao, S. C. Niu, J. Z. Huang,
and M. K. Tan, “Collaborative unsupervised domain adaptation for
medical image diagnosis,” IEEE Trans. Image Process., vol. 29,
pp. 7834–7844, Jul. 2020.
[18] P. Yang, N. X. Xiong, and J. L. Ren, “Data security and privacy
protection for cloud storage: A survey,” IEEE Access, vol. 8,
pp. 131723–131740, Jul. 2020.
[19] C. Zhang, Y. Xie, H. Bai, B. Yu, W. H. Li, and Y. Gao, “A survey on
federated learning,” Knowl.-Based Syst., vol. 216, p. 106775, Mar.
2021.
[20] V. Mothukuri, R. M. Parizi, S. Pouriyeh, Y. Huang, A. Dehghantanha,
and G. Srivastava, “A survey on security and privacy of federated
learning,” Future Gener. Comput. Syst., vol. 115, pp. 619–640, Feb.
2021.
[21] V. Kulkarni, M. Kulkarni, and A. Pant, “Survey of personalization
techniques for federated learning,” in Proc. 4th World Conf. Smart
Trends in Systems, Security and Sustainability, London, UK, 2020, pp.
794−797.
[22] S. Rashidi, M. Mehrad, H. Ghorbani, D. A. Wood, N. Mohamadian, J.
Moghadasi, and S. Davoodi, “Determination of bubble point pressure
& oil formation volume factor of crude oils applying multiple hidden
layers extreme learning machine algorithms,” J. Pet. Sci. Eng.,
vol. 202, p. 108425, Jul. 2021.
[23] G. Vilone and L. Longo, “Notions of explainability and evaluation
approaches for explainable artificial intelligence,” Inf. Fusion, vol. 76,
pp. 89–106, Dec. 2021.
[24] W. J. Murdoch, C. Singh, K. Kumbier, R. Abbasi-Asl, and B. Yu,
“Definitions, methods, and applications in interpretable machine
learning,” Proc. Natl. Acad. Sci. USA, vol. 116, no. 44, pp. 22071–
22080, Oct. 2019.
[25] A. B. Arrieta, N. Díaz-Rodríguez, J. Del Ser, A. Bennetot, S. Tabik, A.
Barbado, S. Garcia, S. Gil-Lopez, D. Molina, R. Benjamins, R.
870
Chatila, and F. Herrera, “Explainable artificial intelligence (XAI):
Concepts, taxonomies, opportunities and challenges toward
responsible AI,” Inf. Fusion, vol. 58, pp. 82–115, Jun. 2020.
[26] A. Holzinger and H. Müller, “Toward human–AI interfaces to support
explainability and causability in medical AI,” Computer, vol. 54,
no. 10, pp. 78–86, Oct. 2021.
[27] R. Guidotti, A. Monreale, S. Ruggieri, F. Turini, F. Giannotti, and D.
Pedreschi, “A survey of methods for explaining black box models,”
ACM Comput. Surv., vol. 51, no. 5, p. 93, Sept. 2019.
[28] A. Mohanty and S. Mishra, “A comprehensive study of explainable
artificial intelligence in healthcare,” in Augmented Intelligence in
Healthcare: A Pragmatic and Integrated Analysis, S. Mishra, H. K.
Tripathy, P. Mallick, and K. Shaalan, Eds. Singapore, Singapore:
Springer, 2022, pp. 475−502.
[29] G. Csurka, “A comprehensive survey on domain adaptation for visual
applications,” in Domain Adaptation in Computer Vision Applications,
G. Csurka, Ed. Cham, Germany: Springer, 2017, pp. 1−35.
[30] P. Koniusz, Y. Tas, H. G. Zhang, M. Harandi, F. Porikli, and R.
Zhang, “Museum exhibit identification challenge for the supervised
domain adaptation and beyond,” in Proc. 15th European Conf.
Computer Vision, Munich, Germany, 2018, pp. 815−833.
[31] R. G. Praveen, E. Granger, and P. Cardinal, “Deep weakly supervised
domain adaptation for pain localization in videos,” in Proc. 15th IEEE
Int. Conf. Automatic Face and Gesture Recognition, Buenos Aires,
Argentina, 2020, pp. 473−480.
[32] L. L. Guo, S. R. Pfohl, J. Fries, A. E. W. Johnson, J. Posada, C.
Aftandilian, N. Shah, and L. Sung, “Evaluation of domain
generalization and adaptation on improving model robustness to
temporal dataset shift in clinical medicine,” Sci. Rep., vol. 12, no. 1, p.
2726, Feb. 2022.
[33] Y. F. Zhang, S. C. Niu, Z. Qiu, Y. Wei, P. L. Zhao, J. H. Yao, J. Z.
Huang, Q. Y. Wu, and M. K. Tan, “COVID-DA: Deep domain
adaptation from typical pneumonia to COVID-19,” arXiv preprint
arXiv: 2005.01577, 2020.
[34] M. Kim and H. Byun, “Learning texture invariant representation for
domain adaptation of semantic segmentation,” in Proc. IEEE/CVF
Conf. Computer Vision and Pattern Recognition, Seattle, USA, 2020,
pp. 12972−12981.
[35] W. J. Yan, Y. Y. Wang, S. J. Gu, L. Huang, F. H. Yan, L. M. Xia, and
Q. Tao, “The domain shift problem of medical image segmentation
and vendor-adaptation by Unet-GAN,” in Proc. 22nd Int. Conf.
Medical Image Computing and Computer-Assisted Intervention,
Shenzhen, China, 2019, pp. 623−631.
[36] R. Argelaguet, D. Arnol, D. Bredikhin, Y. Deloro, B. Velten, J. C.
Marioni, and O. Stegle, “MOFA+: A statistical framework for
comprehensive integration of multi-modal single-cell data,” Genome
Biol., vol. 21, no. 1, p. 111, May 2020.
[37] J. L. Yang, N. C. Dvornek, F. Zhang, J. Chapiro, M. D. Lin, and J. S.
Duncan, “Unsupervised domain adaptation via disentangled
representations: Application to cross-modality liver segmentation,” in
Proc. 22nd Int. Conf. Medical Image Computing and Computer-
Assisted Intervention, Shenzhen, China, 2019, pp. 255−263.
[38] C. Chen, Q. Dou, H. Chen, J. Qin, and P. A. Heng, “Unsupervised
bidirectional cross-modality adaptation via deeply synergistic image
and feature alignment for medical image segmentation,” IEEE Trans.
Med. Imaging, vol. 39, no. 7, pp. 2494–2505, Jul. 2020.
[39] A. J. Larrazabal, N. Nieto, V. Peterson, D. H. Milone, and E. Ferrante,
“Gender imbalance in medical imaging datasets produces biased
classifiers for computer-aided diagnosis,” Proc. Natl. Acad. Sci. USA,
vol. 117, no. 23, pp. 12592–12594, May 2020.
[40] M. N. Du, F. Yang, N. Zou, and X. Hu, “Fairness in deep learning: A
computational perspective,” IEEE Intell. Syst., vol. 36, no. 4, pp. 25–34,
Jul.–Aug. 2021.
[41] Y. Kim, D. Cho, K. Han, P. Panda, and S. Hong, “Domain adaptation
without source data,” IEEE Trans. Artif. Intell., vol. 2, no. 6, pp. 508–
518, Dec. 2021.
[42] S. C. Zhao, B. Li, C. Reed, P. F. Xu, and K. Keutzer, “Multi-source
domain adaptation in the deep learning era: A systematic survey,”
arXiv preprint arXiv: 2002.12169, 2020.
IEEE/CAA JOURNAL OF AUTOMATICA SINICA, VOL. 10, NO. 4, APRIL 2023
[43] H. Wang, M. H. Xu, B. B. Ni, and W. J. Zhang, “Learning to combine:
Knowledge aggregation for multi-source domain adaptation,” in Proc.
16th European Conf. Computer Vision, Glasgow, UK, 2020, pp.
727−744.
[44] S. C. Zhao, X. Y. Yue, S. H. Zhang, B. Li, H. Zhao, B. C. Wu, R.
Krishna, J. E. Gonzalez, A. L. Sangiovanni-Vincentelli, S. A. Seshia,
and K. Keutzer, “A review of single-source deep unsupervised visual
domain adaptation,” IEEE Trans. Neural Netw. Learning Syst., vol. 33,
no. 2, pp. 473–493, Feb. 2022.
[45] J. N. Kundu, N. Venkat, M. V. Rahul, and R. V. Babu, “Universal
source-free domain adaptation,” in Proc. IEEE/CVF Conf. Computer
Vision and Pattern Recognition, Seattle, USA, 2020, pp. 4543−4552.
[46] Y. Liu, W. Zhang, and J. Wang, “Source-free domain adaptation for
semantic segmentation,” in Proc. IEEE/CVF Conf. Computer Vision
and Pattern Recognition, Nashville, USA, 2021, pp. 1215−1224.
[47] A. Jiménez-Sánchez, M. Tardy, M. A. G. Ballester, D. Mateus, and G.
Piella, “Memory-aware curriculum federated learning for breast cancer
classification,” Comput. Methods Programs Biomed., vol. 229, p.
107318, Feb. 2023.
[48] M. Jiménez-Guarneros and P. Gómez-Gil, “A study of the effects of
negative transfer on deep unsupervised domain adaptation methods,”
Expert Syst. Appl., vol. 167, p. 114088, Apr. 2021.
[49] L. T. Feng, F. Qian, X. He, Y. Q. Fan, H. P. Cai, and G. M. Hu,
“Transitive transfer sparse coding for distant domain,” in Proc. IEEE
Int. Conf. Acoustics, Speech and Signal Processing, Toronto, Canada,
2021, pp. 3165−3169.
[50] M. Wang and W. H. Deng, “Deep visual domain adaptation: A
survey,” Neurocomputing, vol. 312, pp. 135–153, Oct. 2018.
[51] M. J. Sheller, B. Edwards, G. A. Reina, J. Martin, S. Pati, A.
Kotrotsou, M. Milchenko, W. L. Xu, D. Marcus, R. R. Colen, and S.
Bakas, “Federated learning in medicine: Facilitating multi-institutional
collaborations without sharing patient data,” Sci. Rep., vol. 10, no. 1, p.
12598, Jul. 2020.
[52] K. V. Sarma, S. Harmon, T. Sanford, H. R. Roth, Z. Y. Xu, J.
Tetreault, D. G. Xu, M. G. Flores, A. G. Raman, R. Kulkarni, B. J.
Wood, P. L. Choyke, A. M. Priester, L. S. Marks, S. S. Raman, D.
Enzmann, B. Turkbey, W. Speier, and C. W. Arnold, “Federated
learning improves site performance in multicenter deep learning
without data sharing,” J. Am. Med. Inf. Assoc., vol. 28, no. 6,
pp. 1259–1264, Jun. 2021.
[53] R. Kumar, A. A. Khan, J. Kumar, Zakria, N. A. Golilarz, S. M. Zhang,
Y. Ting, C. Y. Zheng, and W. Y. Wang, “Blockchain-federated-
learning and deep learning models for COVID-19 detection using CT
imaging,” IEEE Sens. J., vol. 21, no. 14, pp. 16301–16314, Jul. 2021.
[54] M. G. Crowson, D. Moukheiber, A. R. Arévalo, B. D. Lam, S.
Mantena, A. Rana, D. Goss, D. W. Bates, and L. A. Celi, “A
systematic review of federated learning applications for biomedical
data,” PLOS Digit. Health, vol. 1, no. 5, p. e0000033, May 2022.
[55] D. McGraw and K. D. Mandl, “Privacy protections to encourage use of
health-relevant digital data in a learning health system,” NPJ Digit.
Med., vol. 4, no. 1, p. 2, Jan. 2021.
[56] C. H. Wu, F. Z. Wu, L. J. Lyu, Y. F. Huang, and X. Xie,
“Communication-efficient federated learning via knowledge
distillation,” Nat. Commun., vol. 13, no. 1, p. 2032, Apr. 2022.
[57] L. Q. Qu, Y. Y. Zhou, P. P. Liang, Y. D. Xia, F. F. Wang, E. Adeli, L.
Fei-Fei, and D. Rubin, “Rethinking architecture design for tackling
data heterogeneity in federated learning,” in Proc. IEEE/CVF Conf.
Computer Vision and Pattern Recognition, New Orleans, USA, 2022,
pp. 10051−10061.
[58] G. J. Annas, “HIPAA regulations–A new era of medical-record
privacy?” N. Engl. J. Med., vol. 348, pp. 1486–1490, Apr. 2003.
[59] P. Voigt and A. Von dem Bussche, The EU General Data Protection
Regulation (GDPR). A Practical Guide. Cham, Germany: Springer,
2017.
[60] S. Rajendran, J. S. Obeid, H. Binol, R. Jr. D’Agostino, K. Foley, W.
Zhang, P. Austin, J. Brakefield, M. N. Gurcan, and U. Topaloglu,
“Cloud-based federated learning implementation across medical
centers,” JCO Clin. Cancer Inf., vol. 5, pp. 1–11, Jan. 2021.
[61] J. W. Kang, Z. H. Xiong, D. Niyato, Y. Z. Zou, Y. Zhang, and M.
CHADDAD et al.: EXPLAINABLE, DOMAIN-ADAPTIVE, AND FEDERATED ARTIFICIAL INTELLIGENCE IN MEDICINE
Guizani, “Reliable federated learning for mobile networks,” IEEE
Wirel. Commun., vol. 27, no. 2, pp. 72–80, Apr. 2020.
[62] K. A. Bonawitz, H. Eichner, W. Grieskamp, D. Huba, A. Ingerman, V.
Ivanov, C. Kiddon, J. Konecný, S. Mazzocchi, B. McMahan, T. Van
Overveldt, D. Petrou, D. Ramage, J. Roselander, “Towards federated
learning at scale: System design,” in Proc. Machine Learning and
Systems, Stanford, USA, 2019, pp. 374−388.
[63] T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated learning:
Challenges, methods, and future directions,” IEEE Signal Process.
Mag., vol. 37, no. 3, pp. 50–60, May 2020.
[64] J. Xu, B. S. Glicksberg, C. Su, P. Walker, J. Bian, and F. Wang,
“Federated learning for healthcare informatics,” J. Healthc. Inform.
Res., vol. 5, no. 1, pp. 1–19, Nov. 2021.
[65] Y. Zhao, M. Li, L. Z. Lai, N. Suda, D. Civin, and V. Chandra,
“Federated learning with non-IID data,” arXiv preprint arXiv:
1806.00582, 2018.
[66] A. Fallah, A. Mokhtari, and A. Ozdaglar, “Personalized federated
learning: A meta-learning approach,” arXiv preprint arXiv:
2002.07948, 2020.
[67] G. H. Lee and S. Y. Shin, “Federated learning on clinical benchmark
data: Performance assessment,” J. Med. Internet Res., vol. 22, no. 10, p.
e20891, Oct. 2020.
[68] H. R. Roth, K. Chang, P. Singh, N. Neumark, W. Q. Li, V. Gupta, S.
Gupta, L. Q. Qu, A. Ihsani, B. C. Bizzo, Y. H. Wen, V. Buch, M.
Shah, F. Kitamura, M. Mendonça, V. Lavor, A. Harouni, C. Compas,
J. Tetreault, P. Dogra, Y. Cheng, S. Erdal, R. White, B. Hashemian, T.
Schultz, M. Zhang, A. McCarthy, B. M. Yun, E. Sharaf, K. V. Hoebel,
J. B. Patel, B. Chen, S. Ko, E. Leibovitz, E. D. Pisano, L. Coombs, D.
G. Xu, K. J. Dreyer, I. Dayan, R. C. Naidu, M. Flores, D. Rubin, and J.
Kalpathy-Cramer, “Federated learning for breast density classification:
A real-world implementation,” in Proc. 2nd MICCAI Workshop on
Domain Adaptation and Representation Transfer, Lima, Peru, 2020,
pp. 181−191.
[69] G. Joshi, A. Srivastava, B. Yagnik, M. Hasan, Z. Saiyed, L. A.
Gabralla, A. Abraham, R. Walambe, and K. Kotecha, “Explainable
misinformation detection across multiple social media platforms,”
arXiv preprint arXiv: 2203.11724, 2022.
[70] A. Raza, K. P. Tran, L. Koehl, and S. J. Li, “Designing ECG
monitoring healthcare system with federated transfer learning and
explainable AI,” Knowl.-Based Syst., vol. 236, p. 107763, Jan. 2022.
[71] X. C. Peng, Z. J. Huang, Y. Z. Zhu, and K. Saenko, “Federated
adversarial domain adaptation,” in Proc. 8th Int. Conf. Learning
Representations, Addis Ababa, Ethiopia, 2020.
[72] X. X. Li, Y. F. Gu, N. Dvornek, L. H. Staib, P. Ventola, and J. S.
Duncan, “Multi-site fMRI analysis using privacy-preserving federated
learning and domain adaptation: ABIDE results,” Med. Image Anal.,
vol. 65, p. 101765, Oct. 2020.
[73] A. Holzinger, A. Saranti, C. Molnar, P. Biecek, and W. Samek,
“Explainable AI methods-a brief overview,” in Proc. Int. Workshop on
Extending Explainable AI Beyond Deep Models and Classifiers,
Vienna, Austria, 2022, pp. 13−38.
[74] J. T. Springenberg, A. Dosovitskiy, T. Brox, and M. Riedmiller,
“Striving for simplicity: The all convolutional net,” in Proc. 3rd Int.
Conf. Learning Representations, San Diego, USA, 2015.
[75] S. Heidarian, P. Afshar, N. Enshaei, F. Naderkhani, M. J. Rafiee, F. B.
Fard, K. Samimi, S. F. Atashzar, A. Oikonomou, K. N. Plataniotis, and
A. Mohammadi, “COVID-FACT: A fully-automated capsule network-
based framework for identification of COVID-19 cases from chest CT
scans,” Front. Artif. Intell., vol. 4, p. 598932, Feb. 2021.
[76] A. Chaddad, M. L. Zhang, C. Desrosiers, and T. Niazi, “Deep radiomic
features from MRI scans predict survival outcome of recurrent
glioblastoma,” in Proc. 1st Int. Workshop on Radiomics and
Radiogenomics in Neuro-Oncology, Shenzhen, China, 2019, pp.
36−43.
[77] A. Chaddad, C. Desrosiers, B. Abdulkarim, and T. Niazi, “Predicting
the gene status and survival outcome of lower grade glioma patients
with multimodal MRI features,” IEEE Access, vol. 7, pp. 75976–75984,
Jun. 2019.
[78] A. Chaddad, P. Sargos, and C. Desrosiers, “Modeling texture in deep
3D CNN for survival analysis,” IEEE J. Biomed. Health Inform.,
871
vol. 25, no. 7, pp. 2454–2462, Jul. 2021.
[79] A. Chaddad, M. L. Zhang, L. Hassan, and T. Niazi, “Modeling of
textures to predict immune cell status and survival of brain tumour
patients,” in Proc. IEEE 18th Int. Symp. Biomedical Imaging, Nice,
France, 2021, pp. 1067−1071.
[80] A. Chaddad, L. Hassan, and C. Desrosiers, “Deep radiomic analysis
for predicting coronavirus disease 2019 in computerized tomography
and X-ray images,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33,
no. 1, pp. 3–11, Jan. 2022.
[81] A. Chaddad, P. Daniel, M. L. Zhang, S. Rathore, P. Sargos, C.
Desrosiers, and T. Niazi, “Deep radiomic signature with immune cell
markers predicts the survival of glioma patients,” Neurocomputing,
vol. 469, pp. 366–375, Jan. 2022.
[82] S. K. Yeom, P. Seegerer, S. Lapuschkin, A. Binder, S. Wiedemann, K.
R. Müller, and W. Samek, “Pruning by explaining: A novel criterion
for deep neural network pruning,” Pattern Recognit., vol. 115, p.
107899, Jul. 2021.
[83] J. B. Li, C. Q. Zhang, J. T. Zhou, H. Z. Fu, S. Y. Xia, and Q. H. Hu,
“Deep-LIFT: Deep label-specific feature learning for image
annotation,” IEEE Trans. Cybern., vol. 52, no. 8, pp. 7732–7741, Aug.
2022.
[84] A. Shrikumar, P. Greenside, and A. Kundaje, “Learning important
features through propagating activation differences,” in Proc. 34th Int.
Conf. Machine Learning, Sydney, Australia, 2017, pp. 3145−3153.
[85] M. Lin, Q. Chen, and S. C. Yan, “Network in network,” arXiv preprint
arXiv: 1312.4400, 2013.
[86] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D.
Batra, “Grad-CAM: Visual explanations from deep networks via
gradient-based localization,” in Proc. IEEE Int. Conf. Computer
Vision, Venice, Italy, 2017, pp. 618−626.
[87] H. F. Wang, Z. F. Wang, M. N. Du, F. Yang, Z. J. Zhang, S. R. Ding,
P. Mardziel, and X. Hu, “Score-CAM: Score-weighted visual
explanations for convolutional neural networks,” in Proc. IEEE/CVF
Conf. Computer Vision and Pattern Recognition Workshops, Seattle,
USA, 2020, pp. 111−119.
[88] R. Dazeley, P. Vamplew, C. Foale, C. Young, S. Aryal, and F. Cruz,
“Levels of explainable artificial intelligence for human-aligned
conversational explanations,” Artif. Intell., vol. 299, p. 103525, Oct.
2021.
[89] J. Jiménez-Luna, M. Skalic, N. Weskamp, and G. Schneider,
“Coloring molecules with explainable artificial intelligence for
preclinical relevance assessment,” J. Chem. Inf. Model., vol. 61, no. 3,
pp. 1083–1094, Feb. 2021.
[90] S. El-Sappagh, J. M. Alonso, S. M. R. Islam, A. M. Sultan, and K. S.
Kwak, “A multilayer multimodal detection and prediction model based
on explainable artificial intelligence for Alzheimer’s disease,” Sci.
Rep., vol. 11, no. 1, p. 2660, Jan. 2021.
[91] J. F. Peng, K. Q. Zou, M. Zhou, Y. Teng, X. Y. Zhu, F. F. Zhang, and
J. Xu, “An explainable artificial intelligence framework for the
deterioration risk prediction of hepatitis patients,” J. Med. Syst.,
vol. 45, no. 5, p. 61, Apr. 2021.
[92] K. Davagdorj, J. W. Bae, V. H. Pham, N. Theera-Umpon, and K. H.
Ryu, “Explainable artificial intelligence based framework for non-
communicable diseases prediction,” IEEE Access, vol. 9,
pp. 123672–123688, Sept. 2021.
[93] C. Dindorf, J. Konradi, C. Wolf, B. Taetz, G. Bleser, J. Huthwelker, F.
Werthmann, E. Bartaguiz, J. Kniepert, P. Drees, U. Betz, and M.
Fröhlich, “Classification and automated interpretation of spinal posture
data using a pathology-independent classifier and explainable artificial
intelligence (XAI),” Sensors, vol. 21, no. 18, p. 6323, Sept. 2021.
[94] Z. Papanastasopoulos, R. K. Samala, H. P. Chan, L. Hadjiiski, C.
Paramagul, M. A. Helvie, and C. H. Neal, “Explainable AI for medical
imaging: Deep-learning CNN ensemble for classification of estrogen
receptor status from breast MRI,” in Proc. SPIE 11314, Medical
Imaging 2020: Computer-Aided Diagnosis, Houston, Texas, United
States, pp. 113140Z.
[95] R. K. Singh, R. Pandey, and R. N. Babu, “COVIDScreen: Explainable
deep learning framework for differential diagnosis of COVID-19 using
chest X-rays,” Neural Comput. Appl., vol. 33, no. 14, pp. 8871–8892,
Jan. 2021.
872
[96] H. C. Thorsen-Meyer, A. B. Nielsen, A. P. Nielsen, B. S. Kaas-
Hansen, P. Toft, J. Schierbeck, T. Strøm, P. J. Chmura, M. Heimann,
L. Dybdahl, L. Spangsege, P. Hulsen, K. Belling, S. Brunak, and A.
Perner, “Dynamic and explainable machine learning prediction of
mortality in patients in the intensive care unit: A retrospective study of
high-frequency data in electronic patient records,” Lancet Digit.
Health, vol. 2, no. 4, pp. e179–e191, Apr. 2020.
[97] R. Gu, G. T. Wang, T. Song, R. Huang, M. Aertsen, J. Deprest, S.
Ourselin, T. Vercauteren, and S. T. Zhang, “CA-Net: Comprehensive
attention convolutional neural networks for explainable medical image
segmentation,” IEEE Trans. Med. Imaging, vol. 40, no. 2, pp. 699–711,
Feb. 2021.
[98] P. R. Magesh, R. D. Myloth, and R. J. Tom, “An explainable machine
learning model for early detection of Parkinson’s disease using LIME
on DaTSCAN imagery,” Comput. Biol. Med., vol. 126, p. 104041,
Nov. 2020.
[99] R. Karim, T. Döhmen, M. Cochez, O. Beyan, D. Rebholz-Schuhmann,
and S. Decker, “DeepCOVIDExplainer: Explainable COVID-19
diagnosis from chest X-ray images,” in Proc. IEEE Int. Conf.
Bioinformatics and Biomedicine, Seoul, Korea (South), 2020, pp.
1034−1037.
[100] L. A. Jr. de Souza, R. Mendel, S. Strasser, A. Ebigbo, A. Probst, H.
Messmann, J. P. Papa, and C. Palm, “Convolutional neural networks
for the evaluation of cancer in Barrett’s esophagus: Explainable AI to
lighten up the black-box,” Comput. Biol. Med., vol. 135, p. 104578,
Aug. 2021.
[101] T. Dissanayake, T. Fernando, S. Denman, S. Sridharan, H.
Ghaemmaghami, and C. Fookes, “A robust interpretable deep learning
classifier for heart anomaly detection without segmentation,” IEEE J.
Biomed. Health Inform., vol. 25, no. 6, pp. 2162–2171, Jun. 2021.
[102] M. Razzaq, L. Goumidi, M. J. Iglesias, G. Munsch, M. Bruzelius, M.
Ibrahim-Kosta, L. Butler, J. Odeberg, P. E. Morange, and D. A.
Tregouet, “Explainable artificial neural network for recurrent venous
thromboembolism based on plasma proteomics,” in Proc. 19th Int.
Conf. Computational Methods in Systems Biology, Bordeaux, France,
2021, pp. 108−121.
[103] M. La Ferla, M. Montebello, and D. Seychell, “An XAI approach to
deep learning models in the detection of ductal carcinoma in situ,”
arXiv preprint arXiv: 2106.14186, 2021.
[104] C. Gillmann, L. Peter, C. Schmidt, D. Saur, and G. Scheuermann,
“Visualizing multimodal deep learning for lesion prediction,” IEEE
Comput. Graph. Appl., vol. 41, no. 5, pp. 90–98, Sep.–Oct. 2021.
[105] C. Ieracitano, N. Mammone, M. Versaci, G. Varone, A. R. Ali, A.
Armentano, G. Calabrese, A. Ferrarelli, L. Turano, C. Tebala, Z.
Hussain, Z. Sheikh, A. Sheikh, G. Sceni, A. Hussain, and F. C.
Morabito, “A fuzzy-enhanced deep learning approach for early
detection of COVID-19 pneumonia from portable chest X-ray
images,” Neurocomputing, vol. 481, pp. 202–215, Apr. 2022.
[106] J. M. Mayor-Torres, S. Medina-DeVilliers, T. Clarkson, M. D. Lerner,
and G. Riccardi, “Evaluation of interpretability for deep learning
algorithms in EEG emotion recognition: A case study in autism,”
arXiv preprint arXiv: 2111.13208, 2021.
[107] K. H. Kim, H. W. Koo, B. J. Lee, S. W. Yoon, and M. J. Sohn,
“Cerebral hemorrhage detection and localization with medical imaging
for cerebrovascular disease diagnosis and treatment using explainable
deep learning,” J. Korean Phys. Soc., vol. 79, no. 3, pp. 321–327, May
2021.
[108] R. Arefin, M. D. Samad, F. A. Akyelken, and A. Davanian, “Non-
transfer deep learning of optical coherence tomography for post-hoc
explanation of macular disease classification,” in Proc. IEEE 9th Int.
Conf. Healthcare Informatics, Victoria, Canada, 2021, pp. 48−52.
[109] R. T. Sutton, O. R. Zaïane, R. Goebel, and D. C. Baumgart, “Artificial
intelligence enabled automated diagnosis and grading of ulcerative
colitis endoscopy images,” Sci. Rep., vol. 12, no. 1, p. 2748, Feb. 2022.
[110] D. H. Chen, H. J. Zhao, J. R. He, Q. Pan, and W. L. Zhao, “An causal
XAI diagnostic model for breast cancer based on mammography
reports,” in Proc. IEEE Int. Conf. Bioinformatics and Biomedicine,
Houston, USA, 2021, pp. 3341−3349.
[111] Z. Uddin, K. K. Dysthe, A. Følstad, and P. B. Brandtzaeg, “Deep
learning for prediction of depressive symptoms in a large textual
IEEE/CAA JOURNAL OF AUTOMATICA SINICA, VOL. 10, NO. 4, APRIL 2023
dataset,” Neural Comput. Appl., vol. 34, no. 1, pp. 721–744, Jan. 2022.
[112] S. Martínez-Agüero, C. Soguero-Ruiz, J. M. Alonso-Moral, I. Mora-
Jiménez, J. Álvarez-Rodríguez, and A. G. Marques, “Interpretable
clinical time-series modeling with intelligent feature selection for early
prediction of antimicrobial multidrug resistance,” Future Gener.
Comput. Syst., vol. 133, pp. 68–83, Aug. 2022.
[113] K. Wickstrøm, K. Ø. Mikalsen, M. Kampffmeyer, A. Revhaug, and R.
Jenssen, “Uncertainty-aware deep ensembles for reliable and
explainable predictions of clinical time series,” IEEE J. Biomed.
Health Inform., vol. 25, no. 7, pp. 2435–2444, Jul. 2021.
[114] C. Y. Wang, T. S. Ko, and C. C. Hsu, “Machine learning with
explainable artificial intelligence vision for characterization of solution
conductivity using optical emission spectroscopy of plasma in aqueous
solution,” Plasma Process. Polym., vol. 18, no. 12, p. 2100096, Dec.
2021.
[115] Y. G. Akhlaghi, K. Aslansefat, X. D. Zhao, S. Sadati, A. Badiei, X.
Xiao, S. Shittu, Y. Fan, and X. L. Ma, “Hourly performance forecast of
a dew point cooler using explainable artificial intelligence and
evolutionary optimisations by 2050,” Appl. Energy, vol. 281, p.
116062, Jan. 2021.
[116] J. M. Valverde, V. Imani, A. Abdollahzadeh, R. De Feo, M. Prakash,
R. Ciszek, and J. Tohka, “Transfer learning in magnetic resonance
brain imaging: A systematic review,” J. Imaging, vol. 7, no. 4, p. 66,
Apr. 2021.
[117] A. Farahani, S. Voghoei, K. Rasheed, and H. R. Arabnia, “A brief
review of domain adaptation,” in Advances in Data Science and
Information Engineering, R. Stahlbock, G. M. Weiss, M. Abou-Nasr,
C. Y. Yang, H. R. Arabnia, and L. Deligiannidis, Eds. Cham,
Germany: Springer, 2021, pp. 877−894.
[118] D. N. Liu, D. H. Zhang, Y. Song, F. Zhang, L. O’Donnell, H. Huang,
M. Chen, and W. D. Cai, “Unsupervised instance segmentation in
microscopy images via panoptic domain adaptation and task re-
weighting,” in Proc. IEEE/CVF Conf. Computer Vision and Pattern
Recognition, Seattle, USA, 2020, pp. 4242−4251.
[119] A. Choudhary, L. Tong, Y. D. Zhu, and M. D. Wang, “Advancing
medical imaging informatics by deep learning-based domain adap-
tation,” Yearb. Med. Inform., vol. 29, no. 1, pp. 129–138, Aug. 2020.
[120] S. T. Niu, Y. H. Hu, J. Wang, Y. X. Liu, and H. B. Song, “Feature-
based distant domain transfer learning,” in Proc. IEEE Int. Conf. Big
Data, Atlanta, USA, 2020, pp. 5164−5171.
[121] Y. Yang, J. Guo, Q. W. Ye, Y. L. Xia, P. Yang, A. Ullah, and K.
Muhammad, “A weighted multi-feature transfer learning framework
for intelligent medical decision making,” Appl. Soft Comput., vol. 105,
p. 107242, Jul. 2021.
[122] A. Ackaouy, N. Courty, E. Vallée, O. Commowick, C. Barillot, and F.
Galassi, “Unsupervised domain adaptation with optimal transport in
multi-site segmentation of multiple sclerosis lesions from MRI data,”
Front. Comput. Neurosci., vol. l, no. 14, p. 19, Mar. 2020.
[123] X. J. Ma, Y. H. Niu, L. Gu, Y. S. Wang, Y. T. Zhao, J. Bailey, and F.
Lu, “Understanding adversarial attacks on deep learning based medical
image analysis systems,” Pattern Recognit., vol. 110, p. 107332, Feb.
2021.
[124] Y. Z. Sun, G. Yang, D. Y. Ding, G. W. Cheng, J. P. Xu, and X. Ro. Li,
“A GAN-based domain adaptation method for glaucoma diagnosis,” in
Proc. Int. Joint Conf. Neural Networks, Glasgow, UK, 2020, pp. 1−8.
[125] C. M. He, S. M. Wang, H. Y. Kang, L. Q. Zheng, T. F. Tan, and X. J.
Fan, “Adversarial domain adaptation network for tumor image
diagnosis,” Int. J. Approx. Reason., vol. 135, pp. 38–52, Aug. 2021.
[126] G. J. Wang, M. Chen, Z. J. Ding, J. W. Li, H. Z. Yang, and P. Zhang,
“Inter-patient ECG arrhythmia heartbeat classification based on
unsupervised domain adaptation,” Neurocomputing, vol. 454,
pp. 339–349, Sept. 2021.
[127] E. Lee, A. Ho, Y. T. Wang, C. H. Huang, and C. Y. Lee, “Cross-
domain adaptation for biometric identification using
photoplethysmogram,” in Proc. IEEE Int. Conf. Acoustics, Speech and
Signal Processing, Barcelona, Spain, 2020, pp. 1289−1293.
[128] X. S. Bian, X. B. Luo, C. Wang, W. Q. Liu, and X. H. Lin, “DDA-Net:
Unsupervised cross-modality medical image segmentation via dual
domain adaptation,” Comput. Methods Programs Biomed., vol. 213, p.
106531, Jan. 2022.
CHADDAD et al.: EXPLAINABLE, DOMAIN-ADAPTIVE, AND FEDERATED ARTIFICIAL INTELLIGENCE IN MEDICINE
[129] Q. Dou, C. Ouyang, C. Chen, H. Chen, B. Glocker, X. H. Zhuang, and
P. A. Heng, “PnP-AdaNet: Plug-and-play adversarial domain
adaptation network at unpaired cross-modality cardiac segmentation,”
IEEE Access, vol. 7, pp. 99065–99076, Jul. 2019.
[130] H. Guan, L. Wang, and M. X. Liu, “Multi-source domain adaptation
via optimal transport for brain dementia identification,” in Proc. IEEE
18th Int. Symp. Biomedical Imaging, Nice, France, 2021, pp.
1514−1517.
[131] H. Guan, Y. B. Liu, E. K. Yang, P. T. Yap, D. G. Shen, and M. X. Liu,
“Multi-site MRI harmonization via attention-guided deep domain
adaptation for brain disorder identification,” Med. Image Anal., vol. 71,
p. 102076, Jul. 2021.
[132] J. Wang, L. C. Zhang, Q. Wang, L. Chen, J. Shi, X. B. Chen, Z. Y. Li,
and D. G. Shen, “Multi-class ASD classification based on functional
connectivity and functional correlation tensor via multi-source domain
adaptation and multi-view sparse representation,” IEEE Trans. Med.
Imaging, vol. 39, no. 10, pp. 3137–3147, Oct. 2020.
[133] H. F. Cui, C. Yuwen, J. Lei, Y. Xia, and Y. N. Zhang, “Bidirectional
cross-modality unsupervised domain adaptation using generative
adversarial networks for cardiac image segmentation,” Comput. Biol.
Med., vol. 136, p. 104726, Sept. 2021.
[134] R. Xu, Z. Cong, X. C. Ye, S. Kido, and N. Tomiyama, “Unsupervised
content-preserved adaptation network for classification of pulmonary
textures from different CT scanners,” in Proc. IEEE Int. Conf.
Acoustics, Speech and Signal Processing, Barcelona, Spain, 2020, pp.
1060−1064.
[135] P. Wang, P. F. Li, Y. M. Li, J. Xu, and M. F. Jiang, “Classification of
histopathological whole slide images based on multiple weighted semi-
supervised domain adaptation,” Biomed. Signal Process. Control,
vol. 73, p. 103400, Mar. 2022.
[136] Y. Wang, Y. Q. Feng, L. Zhang, Z. Z. Wang, Q. Lv, and Z. Yi, “Deep
adversarial domain adaptation for breast cancer screening from
mammograms,” Med. Image Anal., vol. 73, p. 102147, Oct. 2021.
[137] J. Li, C. L. Feng, X. Z. Lin, and X. H. Qian, “Utilizing GCN and meta-
learning strategy in unsupervised domain adaptation for pancreatic
cancer segmentation,” IEEE J. Biomed. Health Inform., vol. 26, no. 1,
pp. 79–89, Jan. 2022.
[138] M. Saiz-Vivó, A. Colomer, C. Fonfría, L. Martí-Bonmatí, and V.
Naranjo, “Supervised domain adaptation for automated semantic
segmentation of the atrial cavity,” Entropy, vol. 23, no. 7, p. 898, Jul.
2021.
[139] I. Dayan, H. R. Roth, A. X. Zhong, A. Harouni, A. Gentili, A. Z.
Abidin, A. Liu, A. B. Costa, B. J. Wood, C. S. Tsai, C. H. Wang, C. N.
Hsu, C. K. Lee, P. Y. Ruan, D. G. Xu, D. F. Wu, E. Huang, F. C.
Kitamura, G. Lacey, G. C. de Antônio Corradi, G. Nino, H. H. Shin,
H. Obinata, H. Ren, J. C. Crane, J. Tetreault, J. H. Guan, J. W. Garrett,
J. D. Kaggie, J. G. Park, K. Dreyer, K. Juluru, K. Kersten, M. A. B. C.
Rockenbach, M. G. Linguraru, M. A. Haider, M. AbdelMaseeh, N.
Rieke, P. F. Damasceno, P. M. C. e Silva, P. C. Wang, S. Xu, S.
Kawano, S. Sriswasdi, S. Y. Park, T. M. Grist, V. Buch, W.
Jantarabenjakul, W. Wang, W. Y. Tak, X. Li, X. H. Lin, Y. J. Kwon,
A. Quraini, A. Feng, A. N. Priest, B. Turkbey, B. Glicksberg, B.
Bizzo, B. S. Kim, C. Tor-Díez, C. C. Lee, C. J. Hsu, C. N. Lin, C. L.
Lai, C. P. Hess, C. Compas, D. Bhatia, E. K. Oermann, E. Leibovitz,
H. Sasaki, H. Mori, I. Yang, J. H. Sohn, K. N. K. Murthy, L. C. Fu, M.
R. F. de Mendonça, M. Fralick, M. K. Kang, M. Adil, N. Gangai, P.
Vateekul, P. Elnajjar, S. Hickman, S. Majumdar, S. L. McLeod, S.
Reed, S. Gräf, S. Harmon, T. Kodama, T. Puthanakit, T. Mazzulli, V.
L. de Lavor, Y. Rakvongthai, Y. R. Lee, Y. H. Wen, F. J. Gilbert, M.
G. Flores, and Q. Z. Li, “Federated learning for predicting clinical
outcomes in patients with COVID-19,” Nat. Med., vol. 27, no. 10,
pp. 1735–1743, Oct. 2021.
[140] S. Boughorbel, F. Jarray, N. Venugopal, S. Moosa, H. Elhadi, and M.
Makhlouf, “Federated uncertainty-aware learning for distributed
hospital EHR data,” arXiv preprint arXiv: 1910.12191, 2019.
[141] C. X. Tian, H. L. Li, Y. F. Wang, and S. Q. Wang, “Privacy-preserving
constrained domain generalization for medical image classification,”
arXiv preprint arXiv: 2105.08511, 2021.
[142] M. Nasajpour, M. Karakaya, S. Pouriyeh, and R. M. Parizi, “Federated
transfer learning for diabetic retinopathy detection using CNN
architectures,” in Proc. SoutheastCon, Mobile, USA, 2022, pp.
873
655−660.
[143] A. Vaid, S. K. Jaladanki, J. Xu, S. Teng, A. Kumar, S. Lee, S. Somani,
I. Paranjpe, J. K. De Freitas, T. Y. Wanyan, Wanyan, K. W. Johnson,
M. Bicak, E. Klang, Y. J. Kwon, A. Costa, S. Zhao, R. Miotto, A. W.
Charney, E. Böttinger, Z. A. Fayad, G. N. Nadkarni, F. Wang, and B.
S. Glicksberg, “Federated learning of electronic health records to
improve mortality prediction in hospitalized patients with COVID-19:
Machine learning approach,” JMIR Med. Inform., vol. 9, no. 1, p.
e24207, Jan. 2021.
[144] P. F. Guo, P. Y. Wang, J. Y. Zhou, S. S. Jiang, and V. M. Patel,
“Multi-institutional collaborations for improving deep learning-based
magnetic resonance image reconstruction using federated learning,” in
Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition,
Nashville, USA, 2021, pp. 2423−2432.
[145] M. Y. Lu, R. J. Chen, D. H. Kong, J. Lipkova, R. Singh, D. F. K.
Williamson, T. Y. Chen, and F. Mahmood, “Federated learning for
computational pathology on gigapixel whole slide images,” Med.
Image Anal., vol. 76, p. 102298, Feb. 2022.
[146] M. Flores, I. Dayan, H. Roth, A. X. Zhong, A. Harouni, A. Gentili, A.
Abidin, A. Liu, A. Costa, B. Wood, C. S. Tsai, C. H. Wang, C. N. Hsu,
C. K. Lee, C. Ruan, D. G. Xu, D. F. Wu, E. Huang, F. Kitamura, G.
Lacey, G. C. de AntÔnio Corradi, H. H. Shin, H. Obinata, H. Ren, J.
Crane, J. Tetreault, J. H. Guan, J. Garrett, J. G. Park, K. Dreyer, K.
Juluru, K. Kersten, M. A. B. C. Rockenbach, M. Linguraru, M. Haider,
M. AbdelMaseeh, N. Rieke, P. Damasceno, P. M. C. e Silva, P. Wang,
S. Xu, S C. Kawano, S. Sriswasdi, S. Y. Park, T. Grist, V. Buch, W.
Jantarabenjakul, W. Wang, W. Y. Tak, X. Li, X. H. Lin, F. Kwon, F.
Gilbert, J. Kaggie, Q. Z. Li, A. Quraini, A. Feng, A. Priest, B.
Turkbey, B. Glicksberg, B. Bizzo, B. S. Kim, C. Tor-Diez, C. C. Lee,
C. J. Hsu, C. Lin, C. L. Lai, C. Hess, C. Compas, D. Bhatia, E.
Oermann, E. Leibovitz, H. Sasaki, H. Mori, I. Yang, J. H. Sohn, K. N.
K. Murthy, L C. Fu, M. R. F. de Mendonça, M. Fralick, M. K. Kang,
M. Adil, N. Gangai, P. Vateekul, P. Elnajjar, S. Hickman, S.
Majumdar, S. McLeod, S. Reed, S. Graf, S. Harmon, T. Kodama, T.
Puthanakit, T. Mazzulli, V. de Lima Lavor, Y. Rakvongthai, Y. R.
Lee, Y. H. Wen, “Federated learning used for predicting outcomes in
SARS-COV-2 patients,” Res. Sq., DOI: 10.21203/rs.3.rs-126892/v1
[147] B. L. Y. Agbley, J. P. Li, A. U. Haq, E. K. Bankas, S. Ahmad, I. O.
Agyemang, D. Kulevome, W. D. Ndiaye, B. Cobbinah, and S.
Latipova, “Multimodal melanoma detection with federated learning,”
in Proc. 18th Int. Computer Conf. Wavelet Active Media Technology
and Information Processing, Chengdu, China, 2021, pp. 238−244.
[148] M. F. Zhang, Y. N. Wang, and T. Luo, “Federated learning for
arrhythmia detection of Non-IID ECG,” in Proc. IEEE 6th Int. Conf.
Computer and Communications, Chengdu, China, 2020, pp.
1176−1180.
[149] Y. C. Gu, Q. Q. Hu, X. L. Wang, Z. Zhou, and S. X. Lu, “FedACS: An
efficient federated learning method among multiple medical
institutions with adaptive client sampling,” in Proc. 14th Int. Congr.
Image and Signal Processing, BioMedical Engineering and
Informatics, Shanghai, China, 2021, pp. 1−6.
[150] Y. Kang, Y. Liu, and T. J. Chen, “FedMVT: Semi-supervised vertical
federated learning with MultiView training,” arXiv preprint arXiv:
2008.10838, 2020.
[151] W. H. Sun, Y. Q. Chen, X. D. Yang, J. B. Cao, and Y. X. Song,
“FedIO: Bridge inner-and outer-hospital information for perioperative
complications prognostic prediction via federated learning,” in Proc.
IEEE Int. Conf. Bioinformatics and Biomedicine, Houston, USA,
2021, pp. 3215−3221.
[152] A. Linardos, K. Kushibar, S. Walsh, P. Gkontra, and K. Lekadir,
“Federated learning for multi-center imaging diagnostics: A simulation
study in cardiovascular disease,” Sci. Rep., vol. 12, no. 1, p. 3551, Mar.
2022.
[153] N. Q. Dong and I. Voiculescu, “Federated contrastive learning for
decentralized unlabeled medical images,” in Proc. 24th Int. Conf.
Medical Image Computing and Computer-Assisted Intervention,
Strasbourg, France, 2021, pp. 378−387.
[154] Q. Yang, Y. Liu, T. J. Chen, and Y. X. Tong, “Federated machine
learning: Concept and applications,” ACM Trans. Intell. Syst. Technol.,
vol. 10, no. 2, p. 12, Mar. 2019.
[155] J. Lo, T. T. Yu, D. Ma, P. X. Zang, J. P. Owen, Q. Q. Zhang, R. K.
874
Wang, M. F. Beg, A. Y. Lee, Y. L. Jia, and M. V. Sarunic, “Federated
learning for microvasculature segmentation and diabetic retinopathy
classification of OCT data,” Ophthalmol. Sci., vol. 1, no. 4, p. 100069,
Dec. 2021.
[156] B. Gu, A. Xu, Z. Y. Huo, C. Deng, and H. Huang, “Privacy-preserving
asynchronous vertical federated learning algorithms for multiparty
collaborative learning,” IEEE Trans. Neural Netw. Learn. Syst.,
vol. 33, no. 11, pp. 6103–6115, Nov. 2022.
[157] Y. Liu, Y. Kang, C. P. Xing, T. J. Chen, and Q. Yang, “A secure
federated transfer learning framework,” IEEE Intell. Syst., vol. 35,
no. 4, pp. 70–82, Jul.–Aug. 2020.
[158] S. Li, T. X. Cai, and R. Duan, “Targeting underrepresented
populations in precision medicine: A federated transfer learning
approach,” arXiv preprint arXiv: 2108.12112, 2021.
[159] T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith,
“Federated optimization in heterogeneous networks,” in Proc.
Machine Learning and Systems, Austin, USA, 2020, pp. 429−450.
[160] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas,
“Communication-efficient learning of deep networks from
decentralized data,” in Proc. 20th Int. Conf. Artificial Intelligence and
Statistics, Fort Lauderdale, USA, 2017, pp. 1273−1282.
[161] S. U. Stich, “Local SGD converges fast and communicates little,” in
Proc. 7th Int. Conf. Learning Representations, New Orleans, LA,
USA, May, 2019.
[162] F. Sattler, K. R. Müller, and W. Samek, “Clustered federated learning:
Model-agnostic distributed multitask optimization under privacy
constraints,” IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 8,
pp. 3710–3722, Aug. 2021.
[163] O. Marfoq, G. Neglia, A. Bellet, L. Kameni, and R. Vidal, “Federated
multi-task learning under a mixture of distributions,” in Proc. 35th
Advances in Neural Information Processing Systems, 2021, pp.
15434−15447.
[164] A. C. Hauschild, M. Lemanczyk, J. Matschinske, T. Frisch, O.
Zolotareva, A. Holzinger, J. Baumbach, and D. Heider, “Federated
random forests can improve local performance of predictive models
for various healthcare applications,” Bioinformatics, vol. 38, no. 8,
pp. 2278–2286, Apr. 2022.
[165] A. Chowdhury, H. Kassem, N. Padoy, R. Umeton, and A. Karargyris,
“A review of medical federated learning: Applications in oncology and
cancer research,” in Proc. 7th Int. MICCAI Brainlesion Workshop,
2022, pp. 3−24.
[166] J. Deng, W. Dong, R. Socher, L. J. Li, K. Li, and L. Fei-Fei,
“ImageNet: A large-scale hierarchical image database,” in Proc. IEEE
Conf. Computer Vision and Pattern Recognition, Miami, USA, 2009,
pp. 248−255.
[167] Y. Otaki, A. Singh, P. Kavanagh, R. J. H. Miller, T. Parekh, B. K.
Tamarappoo, T. Sharir, A. J. Einstein, M. B. Fish, T. D. Ruddy, P. A.
Kaufmann, A. J. Sinusas, E. J. Miller, T. M. Bateman, S. Dorbala, M.
Di Carli, S. Cadet, J. X. Liang, D. Dey, D. S. Berman, and P. J.
Slomka, “Clinical deployment of explainable artificial intelligence of
SPECT for diagnosis of coronary artery disease,” in JACC:
Cardiovasc. Imaging, vol. 15, no. 6, pp. 1091−1102, Jun. 2022.
[168] W. Y. B. Lim, N. C. Luong, D. T. Hoang, Y. T. Jiao, Y. C. Liang, Q.
Yang, D. Niyato, and C. Y. Miao, “Federated learning in mobile edge
networks: A comprehensive survey,” IEEE Commun. Surv. Tut.,
vol. 22, no. 3, pp. 2031–2063, 2020.
[169] H. Guan and M. X. Liu, “Domain adaptation for medical image
analysis: A survey,” IEEE Trans. Biomed. Eng., vol. 60, no. 3,
pp. 1173–1185, Mar. 2022.
[170] W. W. Zhang, F. Wang, Y. Jiang, Z. F. Xu, S. C. Wu, and Y. H.
Zhang, “Cross-subject EEG-based emotion recognition with deep
domain confusion,” in Proc. 12th Int. Conf. Intelligent Robotics and
Applications, Shenyang, China, 2019, pp. 558−570.
[171] S. T. Niu, M. Liu, Y. X. Liu, J. Wang, and H. B. Song, “Distant
domain transfer learning for medical imaging,” IEEE J. Biomed.
Health Inform., vol. 25, no. 10, pp. 3784–3793, Oct. 2021.
[172] C. Chen, Q. Dou, H. Chen, J. Qin, and P. A. Heng, “Synergistic image
and feature adaptation: Towards cross-modality domain adaptation for
medical image segmentation,” in Proc. AAAI Conf. Artif. Intell., vol.
33, no. 1, pp. 865–872, Jul. 2019.
IEEE/CAA JOURNAL OF AUTOMATICA SINICA, VOL. 10, NO. 4, APRIL 2023
[173] M. Meng, J. H. Hu, Y. Y. Gao, W. Z. Kong, and Z. Z. Luo, “A deep
subdomain associate adaptation network for cross-session and cross-
subject EEG emotion recognition,” Biomed. Signal Process. Control,
vol. 78, p. 103873, Sept. 2022.
[174] Z. K. Tian, D. H. Li, Y. Yang, F. Z. Hou, Z. Y. Yang, Y. Song, and Q.
Gao, “A novel domain adversarial networks based on 3D-LSTM and
local domain discriminator for hearing-impaired emotion recognition,”
IEEE J. Biomed. Health Inform., vol. 27, no. 1, pp. 363–373, Jan. 2023.
[175] V. M. Bashyam, J. Doshi, G. Erus, D. Srinivasan, A. Abdulkadir, A.
Singh, M. Habes, Y. Fan, C. L. Masters, P. Maruff, C. J. Zhuo, H.
Völzke, S. C. Johnson, J. Fripp, N. Koutsouleris, T. D. Satterthwaite,
D. H. Wolf, R. E. Gur, R. C. Gur, J. C. Morris, M. S. Albert, H. J.
Grabe, S. M. Resnick, N. R. Bryan, K. Wittfeld, R. Bülow, D. A.
Wolk, H. C. Shou, I. M. Nasrallah, C. Davatzikos, and The iSTAGING
and PHENOM Consortia, “Deep generative medical image
harmonization for improving cross-site generalization in deep learning
predictors,” J. Magn. Reson. Imaging, vol. 55, no. 3, pp. 908–916, Mar.
2022.
[176] C. I. Bercea, B. Wiestler, D. Rueckert, and S. Albarqouni, “Federated
disentangled representation learning for unsupervised brain anomaly
detection,” Nat. Mach. Intell., vol. 4, no. 8, pp. 685–695, Aug. 2022.
[177] X. Bai, H. C. Wang, L. Y. Ma, Y. C. Xu, J. F. Gan, Z. W. Fan, F.
Yang, K. Ma, J. H. Yang, S. Bai, C. Shu, X. Y. Zou, R. H. Huang, C.
Z. Zhang, X. W. Liu, D. D. Tu, C. O. Xu, W. Q. Zhang, X. Wang, A.
G. Chen, Y. Zeng, D. H. Yang, M. W. Wang, N. Holalkere, N. J.
Halin, I. R. Kamel, J. Wu, X. H. Peng, X. Wang, J. B. Shao, P.
Mongkolwat, J. J. Zhang, W. Y. Liu, M. Roberts, Z. Z. Teng, L. Beer,
L. E. Sanchez, E. Sala, D. L. Rubin, A. Weller, J. Lasenby, C. S.
Zheng, J. M. Wang, Z. Li, C. Schönlieb, and T. Xia, “Advancing
COVID-19 diagnosis with privacy-preserving collaboration in artificial
intelligence,” Nat. Mach. Intell., vol. 3, no. 12, pp. 1081–1089, Dec.
2021.
[178] M. T. Ribeiro, S. Singh, and C. Guestrin, ““Why should I trust you?”:
Explaining the predictions of any classifier,” in Proc. 22nd ACM
SIGKDD Int. Conf. Knowledge Discovery and Data Mining, San
Francisco, USA, 2016, pp. 1135−1144.
[179] Q. Huang, M. Yamada, Y. Tian, D. Singh, and Y. Chang,
“GraphLIME: Local interpretable model explanations for graph neural
networks,” IEEE Trans. Knowl. Data Eng., DOI: 10.1109/TKDE.
2022.3187455
[180] S. M. Lundberg and S. I. Lee, “A unified approach to interpreting
model predictions,” in Proc. 31st Int. Conf. Neural Information
Processing Systems, Long Beach, USA, 2017, pp. 4768−4777.
[181] E. Tzeng, J. Hoffman, N. Zhang, K. Saenko, and T. Darrell, “Deep
domain confusion: Maximizing for domain invariance,” arXiv preprint
arXiv: 1412.3474, 2014.
[182] M. S. Long, Y. Cao, J. M. Wang, and M. I. Jordan, “Learning
transferable features with deep adaptation networks,” in Proc. 32nd
Int. Conf. Machine Learning, Lille, France, 2015, pp. 97−105.
[183] X. T. Han, L. Qi, Q. Yu, Z. Q. Zhou, Y. F. Zheng, Y. H. Shi, and Y.
Gao, “Deep symmetric adaptation network for cross-modality medical
image segmentation,” IEEE Trans. Med. Imaging, vol. 41, no. 1,
pp. 121–132, Jan. 2022.
[184] Y. C. Zhu, F. Z. Zhuang, J. D. Wang, G. L. Ke, J. W. Chen, J. Bian, H.
Xiong, and Q. He, “Deep subdomain adaptation network for image
classification,” IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 4,
pp. 1713–1722, Apr. 2021.
[185] Y. R. Jin, Z. Y. Li, Y. Q. Liu, J. L. Liu, C. J. Qin, L. Q. Zhao, and C.
L. Liu, “Multi-class 12-lead ECG automatic diagnosis based on a
novel subdomain adaptive deep network,” Sci. China Technol. Sci.,
vol. 65, no. 11, pp. 2617–2630, Sept. 2022.
[186] J. D. Wang, Y. Q. Chen, W. J. Feng, H. Yu, M. Y. Huang, and Q.
Yang, “Transfer learning with dynamic distribution adaptation,” ACM
Trans. Intell. Syst. Technol., vol. 11, no. 1, p. 6, Feb. 2020.
[187] B. Xu, K. W. Wu, Y. Wu, J. He, and C. Y. Chen, “Dynamic
adversarial domain adaptation based on multikernel maximum mean
discrepancy for breast ultrasound image classification,” Expert Syst.
Appl., vol. 207, p. 117978, Nov. 2022.
[188] H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-MNIST: A novel image
dataset for benchmarking machine learning algorithms,” arXiv preprint
CHADDAD et al.: EXPLAINABLE, DOMAIN-ADAPTIVE, AND FEDERATED ARTIFICIAL INTELLIGENCE IN MEDICINE
arXiv: 1708.07747, 2017.
[189] K. Saenko, B. Kulis, M. Fritz, and T. Darrell, “Adapting visual
category models to new domains,” in Proc. 11th European Conf.
Computer Vision, Heraklion, Crete, Greece, 2010, pp. 213−226.
[190] B. H. Menze, A. Jakab, S. Bauer, J. Kalpathy-Cramer, K. Farahani, J.
Kirby, Y. Burren, N. Porz, J. Slotboom, R. Wiest, L. Lanczi, E.
Gerstner, M. A. Weber, T. Arbel, B. B. Avants, N. Ayache, P.
Buendia, D. L. Collins, N. Cordier, J. J. Corso, A. Criminisi, T. Das,
H. Delingette, Ç. Demiralp, C. R. Durst, M. Dojat, S. Doyle, J. Festa,
F. Forbes, E. Geremia, B. Glocker, P. Golland, X. T. Guo, A.
Hamamci, K. M. Iftekharuddin, R. Jena, N. M. John, E. Konukoglu, D.
Lashkari, J. A. Mariz, R. Meier, S. Pereira, D. Precup, S. J. Price, T. R.
Raviv, S. M. S. Reza, M. Ryan, D. Sarikaya, L. Schwartz, H. C. Shin,
J. Shotton, C. A. Silva, N. Sousa, N. K. Subbanna, G. Szekely, T. J.
Taylor, O. M. Thomas, N. J. Tustison, G. Unal, F. Vasseur, M.
Wintermark, D. H. Ye, L. Zhao, B. S. Zhao, D. Zikic, M. Prastawa, M.
Reyes, and K. Van Leemput, “The multimodal brain tumor image
segmentation benchmark (BRATS),” IEEE Trans. Med. Imaging,
vol. 34, no. 10, pp. 1993–2024, Oct. 2015.
[191] X. H. Zhuang and J. Shen, “Multi-scale patch and multi-modality
atlases for whole heart segmentation of MRI,” Med. Image Anal.,
vol. 31, pp. 77–87, Jul. 2016.
[192] A. E. Kavur, N. S. Gezer, M. Barış, S. Aslan, P. H. Conze, V. Groza,
D. D. Pham, S. Chatterjee, P. Ernst, S. Özkan, B. Baydar, D. Lachinov,
S. Han, J. Pauli, F. Isensee, M. Perkonigg, R. Sathish, R. Rajan, D.
Sheet, G. Dovletov, O. Speck, A. Nürnberger, K. H. Maier-Hein, G. B.
Akar, G. Ünal, O. Dicle, and M. A. Selver, “CHAOS challenge-
combined (CT-MR) healthy abdominal organ segmentation,” Med
Image Anal, vol. 69, p. 101950, Apr. 2021.
[193] M. H. Yap, G. Pons, J. Martí, S. Ganau, M. Sentís, R. Zwiggelaar, A.
K. Davison, and R. Martí, “Automated breast ultrasound lesions
detection using convolutional neural networks,” IEEE J. Biomed.
Health Inform., vol. 22, no. 4, pp. 1218–1226, Jul. 2018.
[194] J. De Fauw, J. R. Ledsam, B. Romera-Paredes, S. Nikolov, N.
Tomasev, S. Blackwell, H. Askham, X. Glorot, B. O’Donoghue, D.
Visentin, G. van den Driessche, B. Lakshminarayanan, C. Meyer, F.
Mackinder, S. Bouton, K. Ayoub, R. Chopra, D. King, A.
Karthikesalingam, C. O. Hughes, R. Raine, J. Hughes, D. A. Sim, C.
Egan, A. Tufail, H. Montgomery, D. Hassabis, G. Rees, T. Back, P. T.
Khaw, M. Suleyman, J. Cornebise, P. A. Keane, and O. Ronneberger,
“Clinically applicable deep learning for diagnosis and referral in
retinal disease,” Nat. Med., vol. 24, no. 9, pp. 1342–1350, Sept. 2018.
[195] K. E. Stephan, F. Schlagenhauf, Q. J. M. Huys, S. Raman, E. A.
Aponte, K. H. Brodersen, L. Rigoux, R. J. Moran, J. Daunizeau, R. J.
Dolan, K. J. Friston, and A. Heinz, “Computational neuroimaging
strategies for single patient predictions,” NeuroImage, vol. 145,
pp. 180–199, Jan. 2017.
[196] C. Anastasopoulos, S. Yang, M. Pradella, T. A. D’Antonoli, S. Knecht,
J. Cyriac, M. Reisert, E. Kellner, R. Achermann, P. Haaf, B. Stieltjes,
A. W. Sauter, J. Bremerich, G. Sommer, and A. Abdulkadir, “Atri-U:
Assisted image analysis in routine cardiovascular magnetic resonance
volumetry of the left atrium,” J. Cardiovasc. Magn. Reson., vol. 23,
no. 1, p. 133, Nov. 2021.
[197] Y. L. Zhu and X. W. Duan, “Predictive nursing helps improve
treatment efficacy, treatment compliance, and quality of life in
unstable angina pectoris patients,” Am. J. Transl. Res., vol. 13, no. 4,
pp. 3473–3479, Apr. 2021.
[198] N. A. M. Pauzi, Y. B. Wah, S. M. Deni, S. Khatijah, N. A. Rahim, and
Suhartono, “Comparison of single and MICE imputation methods for
missing values: A simulation study,” Pertanika J. Sci. Technol.,
vol. 29, no. 2, pp. 979–998, Apr. 2021.
[199] P. Wyss, D. Ginsbourger, H. C. Shou, et al., “Adaptive data-driven
selection of sequences of biological and cognitive markers in pre-
clinical diagnosis of dementia,” medRxiv, DOI: 10.1101/2021.10.26.
21265515, 2021.
875
Ahmad Chaddad received the master degree (2008)
from Université de Technologie de Compiegne and
the Ph.D. degree (2012) from the University of Lor-
raine, France. He worked for seven years at McGill
University, Canada, École de Technologie Supéri-
eure (ETS), Montreal, The University of Texas MD
Anderson Cancer Center, USA, and Villanova Uni-
versity, USA. In 2020, he joined the School of Artifi-
cial Intelligence, Guilin University of Electronic
Technology, as a Professor. He is an Associate Mem-
ber at LIVIA, ETS (Canada) and LCOMS, University of Lorraine (France).
He has authored more than 80 research papers (60 papers first co-author). His
current research interests include AI and radiomics analysis to improve per-
sonalized medicine strategies, by allowing clinicians to monitor disease in real
time as patients move through treatment. Dr. Chaddad is a member of several
international technical and organizational committees.
Qizong Lu is undergraduate student in School of
Artificial Intelligence, Guilin University of Elec-
tronic Technology. From 2021 to 2022, he works on
the current problems faced the clinical practice. His
current research interests include AI for healthcare
and medical image analysis. He has authored a
research paper about radiomics analysis for autism
spectrum disorder.
Jiali Li is undergraduate student in School of Artifi-
cial Intelligence, Guilin University of Electronic Tech-
nology. She works on artificial intelligence for clini-
cal diagnosis. Her current research interests include
machine learning and biomedical engineering. She
has authored a research paper about radiomics analy-
sis for Autism spectrum disorder.
Yousef Katib finished his residency training pro-
gram in Radiation Oncology and fellowship in Geni-
tourinary and gastrointestinal cancer at McGill Uni-
versity, Canada. He also received the master degree
in science from McGill University. He is currently an
Assistant Professor and Consultant in Radiation-
Oncology at Taibah University, Saudi Arabia. His
clinical activity is oriented toward the management
of prostate tumors and radiomic analysis. He coordi-
nates several prospective and retrospective studies in
prostate cancers.
Reem Kateb is an Assistant Professor at Taibah Uni-
versity in the Department of Information Systems,
Saudi Arabia. She received the M.A.Sc. in informa-
tion system security and Ph.D. in information and
systems engineering at Concordia University, Cana-
da. Her research interests are cybersecurity, cyber-
physical system, IoT, and Secure AI.
Camel Tanougast received the Ph.D. degree in
microelectronic and electronic instrumentation from
the Henri Poincaré University of Nancy, France, in
2001, and the authorization/accreditation to super-
vise research (HDR - Habilitation degree) from the
University of Metz, in 2009. He is currently a Profes-
sor in electronics and embedded systems with the
University of Lorraine, France. He joined the Elec-
tronic Architectures Group, Electronic Instrumenta-
tion Laboratory of Nancy, in 2003, and the Micro-
electronic and Sensor Interface Laboratory of Metz (LICM), in 2008. He has
876
been the Head Research of the networked adaptive and self-organized sys-
tems at LICM. He joined the Laboratory of Design, Optimization and Model-
ing Systems (LCOMS), in 2013. He is currently the Deputy Director and the
ASEC Team Leader of LCOMS for research in microelectronics, communi-
cating embedded systems, and smart sensors. He has authored or coauthored
more than 120 publications and several research books, participated in sev-
eral editorial boards of books and international journals, and has been the
Supervisor of 21 Ph.D. thesis. His research interests include radio communi-
cation, reconfigurable systems and NoCs, design and implementation of real
time processing architectures, optimization, and implementation design, Sys-
tem-on-Chip development, FPGA design, computing vision, radiomic analy-
sis, image processing, cryptography, and the digital television broadcast
(DVB).
Ahmed Bouridane (Senior Member, IEEE) received
an “Ingenieur d’Etat” degree in electronics from
“Ecole Nationale Polytechnique” of Algiers (ENPA),
Algeria, in 1982, an M.Phil. degree in electrical engi-
neering (VLSI design for signal processing) from the
University of Newcastle-Upon-Tyne, U.K., in 1988,
and a Ph.D. degree in electrical engineering (com-
puter vision) from the University of Nottingham,
U.K., in 1992. From 1992 to 1994, he worked as a
research developer in telesurveillance and access
control applications. In 1994, he joined Queen’s University Belfast, U.K., ini-
IEEE/CAA JOURNAL OF AUTOMATICA SINICA, VOL. 10, NO. 4, APRIL 2023
tially as Lecturer in computer architecture and image processing and later on
he was promoted to reader in computer science. In 2009, he joined Northum-
bria University at Newcastle leading the Computational Intelligence and
Visual Computing Lab. His is now a professor of machine intelligence and
director of the Cybersecurity and Data Analytics Research Center at the Uni-
versity of Sharjah, UAE. His research interests are in machine learning with
applications to imaging for forensics and security, quantitative pathology and
biomedical engineering, homeland security and video analytics. He has
authored and co-authored more than 350 publications and three research
books on imaging for forensics and security and biometric security and pri-
vacy.
Ahmed Abdulkadir received the master degree
(2012) in life sciences from the École Polytechnique
Fédérale de Lausanne, Switzerland and the Ph.D.
(2018) in computer engineering from the Albert-Lud-
wigs University Freiburg, Freiburg-im-Breisgau, Ger-
many. He worked as a Researcher at the University
of Bern, Switzerland, and the University of Pennsyl-
vania, USA. He is currently completing a three-year
mobility fellowship granted by the Swiss National
Science Foundation at the Lausanne University Hos-
pital, Switzerland. His work focuses on the use of statistical methods and AI
to study aging and dementia in order to improve the diagnosis of brain disor-
ders.