This article has been accepted for inclusion in a future issue of this journal.
Content is final as presented, with the exception of pagination.
IEEE TRANSACTIONS ON CYBERNETICS
Single/Multi-Source Black-Box Domain Adaption
for Sensor Time Series Data
Lei Ren , Member, IEEE, and Xuejun Cheng , Graduate Student Member, IEEE
Abstract—Unsupervised domain adaption (UDA), which trans-
fers knowledge from a labeled source domain to an unlabeled
target domain, has attracted tremendous attention in many
machine learning applications. Recently, there have been attempts
to apply domain adaption for sensor time series data, such as
human activity recognition and gesture recognition. However,
existing methods suffer from some drawbacks that hinder fur-
ther performance improvement. They often require access to
source data or source models during training, which is unavail-
able in some fields because of privacy protection and storage
limit. Typically, the source domains may only provide an appli-
cation programming interface (API) for the target domain to call.
On the other hand, current UDA methods have not considered
the temporal consistency and low-signal-to-noise ratio (SNR) of
sensor time series. To address the challenges, this article presents
a black-box domain adaption framework for sensor time series
data (B2TSDA). First, we propose a single/multi-source teacher–
student learning framework to distill the knowledge from the
source domains to a customized target model. Then we design a
new temporal consistency loss by combining an adaptive mask
method and dynamic threshold method to maintain consistent
temporal information and balance the learning difficulties of dif-
ferent classes. For the multisource black-box domain adaption,
we further propose a Shapley-enhanced method to determine
the contribution of each source domain. Experimental results on
both single-source and multisource domain adaption show that
our framework has superior performance compared to other
black-box UDA methods.
Index Terms—Activity recognition, gesture recognition, mul-
tisource transfer learning, time series, unsupervised domain
adaptation.
I. I NTRODUCTION
ULTISENSOR technologies in the Internet of Things
M have been applied in many fields, such as activity
recognition [1], cyber-security [2], and clinical diagnosis [3].
Sensors can generate a large amount of multivariate time-series
data. Recent trends in artificial intelligence technologies have
Manuscript received 27 May 2023; revised 28 June 2023; accepted 23 July
2023. This work was supported by the National Natural Science Foundation
of China under Grant 62225302, Grant 92167108, and Grant 62173023. This
article was recommended by Associate Editor H. Han. (Corresponding author:
Lei Ren.)
Lei Ren is with the School of Automation Science and Electrical
Engineering, Beihang University, Beijing 100191, China, also with
Zhongguancun Laboratory, Beijing 100194, China, and also with the State Key
Laboratory of Intelligent Manufacturing System Technology, Beijing 100854,
China (e-mail: renlei@buaa.edu.cn).
Xuejun Cheng is with the School of Automation Science and
Electrical Engineering, Beihang University, Beijing 100191, China (e-mail:
lukesoldier@163.com).
Color versions of one or more figures in this article are available at
https://doi.org/10.1109/TCYB.2023.3300832.
Digital Object Identifier 10.1109/TCYB.2023.3300832
2168-2267 
c 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on December 07,2023 at 14:44:18 UTC from IEEE Xplore. Restrictions apply.
1
led to a proliferation of studies that how to apply these new
technologies to the field of industrial environment, including
fault diagnosis [4] and quality prediction [5]. Traditional sen-
sor time series methods usually rely on the feature engineer
or data preprocessing, and their generalization performance
is poor. Recently, with the rise in deep learning, deep neu-
ral networks for sensor time series have successfully achieved
promising performances in these applications. Several success-
ful deep learning models have been proposed for time-series
classification. Different from the interpretable feature-based
representations, various deep learning models are proposed to
capture the spatial interactions in sensor time series. There
are three main encoder architectures incorporating tempo-
ral information in time series-convolutional neural network
(CNN)-based model, recurrent neural network (RNN)-based
model, and attention-based model. CNNs extract local rela-
tionships that are invariant across spatial dimensions. RNNs
extract different range temporal dependencies in the sensor
time series. Attention mechanisms weigh temporal features to
focus on significant time steps in the time-series. However,
these models usually need a large amount of labeled training
data. To tackle the issue of expensive data annotation, unsuper-
vised domain adaption (UDA) models are applied to the sensor
time series. UDA aims to close the gap caused by domain shift
with the help of a small amount of unlabeled data from the
target domain. Generally, UDA minimizes the empirical risk
on the source domain data and generates the domain-invariant
representations [6].
Despite recent advances of deep learning-based approaches
in domain adaption for sensor time series data [7], [8], we
summarize three key challenges in these circumstances. The
first challenge is that the source domain models and source
domain data are not always accessible due to limitations of
data-privacy rules or computation resources [9], although most
domain adaption methods require access to the source data
during adaptation. For example, the information of clients
from the different companies is sensitive and not be allowed
to share, where an application programming interface (API)
service is provided to customers instead. Considering the
data-privacy rules, current UDA methods, such as domain
adversarial training [10], are not suitable. Regarding the com-
putation resources, networks for the target domain adopt the
same networks as the source domains [11], which is inconve-
nient especially when the source domain model is large and
complex. The second challenge is that two particular properties
of sensor time series data are not considered. The sensor time
series data is collected from the industrial machines. So the
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
2 IEEE TRANSACTIONS ON CYBERNETICS
Fig. 1. Description of the back-box domain adaption problems. The source
domains only provide information through black-box predictors, where both
source domain data and models are not acquired during the domain adaption
process.
sensor time series data have significant temporal consistency
and low signal-to-noise ratio (SNR). The existing domain
adaption methods are mainly devised for the image or tex-
tual data. Trivially adopting UDA methods [12], [13] for time
series data may break the temporal consistency and therefore
harm the performance of the model. The third challenge is
the negative transfer effect in multisource domain adaption.
The irrelevant or malicious source domains may cause nega-
tive transfer [14], since we cannot verify whether the source
domain and target domain data share a similar distribution.
To tackle these challenges, we propose black-box domain
adaption for time series data (B2TSDA) to consider the model-
agnostic properties, data-privacy, temporal-consistency, and
low SNR. The contribution of this article is summarized as
follows.
1) Single/Multi-Source Teacher–Student Learning: The
proposed “Black-Box Time-series UDA” framework
is shown in Fig. 1, where both source domain time
series and models are not accessed. We then introduce
single/multi-source teacher–student learning to distill the
knowledge from the provided source domain black-box
predictors to a customized target model. Teacher–student
learning is model-agnostic and is better for obtaining
domain invariant representation, especially when the
source data cannot be directly accessed.
2) Temporal Consistency Loss Combining Adaptive
Masking and Dynamic Threshold: Considering the
characteristics of the time series data, we introduce a
temporal consistency loss mixing up adaptive masking
and dynamic threshold to take the temporal information
into consideration. We use pseudo-learning [15] with a
dynamic threshold to select domain-invariant samples
from the target domain. Then we design a Bernoulli
masking method to leverage the time series information
and improve model robustness.
3) Shapley-Enhanced Method: We propose a Shapley-
enhanced method to address the negative transfer
Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on December 07,2023 at 14:44:18 UTC from IEEE Xplore. Restrictions apply.
problem in the multisource domain adaption, which
conducts a model aggregation scheme. We define the
confidence score to control the penalty degree for the
negative transfer. Then we aggregate a series of source
data time series and set weights according to their
relevance to the target domain time series.
The remainder of this article is structured as follows.
Section II describes the related work, including the single-
source domain adaption, multisource domain adaption, source-
free domain adaption, black-box domain adaption, federated
domain adaption, and teacher–student learning. Section III
introduces the proposed black-box domain adaption frame-
work for time series data, followed by the details for each
part of this method. Section IV depicts the experiments on
single-source and multisource time-series datasets to show the
effectiveness of the proposed approach. Section V concludes
this article and discusses the future work.
II. R ELATED W ORK
In this section, related work, including UDA and teacher–
student learning for time series data is briefly introduced
below.
A. Unsupervised Domain Adaption for Sensor Time Series
Data
Single-Source Domain Adaption: Most single-source UDA
methods attempt to minimize domain discrepancy between
domains to learn a domain invariant space. They minimize
domain discrepancy by utilizing metric-based learning and
adversarial learning. Metric-based learning designs a specific
metric, such as Gaussian kernel maximum mean discrep-
ancy [16] and higher-order moment matching [17], which
can measure the discrepancy distance to learn domain invari-
ant properties. Adversarial learning constructs an adversarial
loss to train discriminators to distinguish between source and
target domains [18]. Ganin et al. [6] proposed a domain-
adversarial neural network to achieve domain adaption.
Moreover, domain separation [19], Wasserstein distance [20],
collaborative network [21], and few-shot learning [22] com-
bined with adversarial learning are applied furthermore to
improve the performance of the adversarial learning model.
Because of the nonstationary and nonmonotonic properties of
the time series, direct metric-based learning and adversarial
learning methods are not suitable for domain adaption for
time series. Zhao et al. [23] proposed deep representation-
based domain adaptation to solve the domain adaption for
time series data. Ragab et al. [24] addressed the time-series
domain adaption with self-supervised autoregressive domain
adaptation. Different from those approaches, we explore spe-
cial situations where neither source domain data nor source
models are not accessible.
Multisource Domain Adaption: Compared to sing-source
domain adaption for time series data, there are limited
researches on multisource domain adaptation, especially time
series domain adaptation. MDAN [25] optimizes task-adaptive
generalization bound using adversarial neural networks.
Xie et al. [26] adopted multiclass domain classifier to
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

REN AND CHENG: SINGLE/MULTI-SOURCE BLACK-BOX DOMAIN ADAPTION FOR SENSOR TIME SERIES DATA 3

Fig. 2. Our proposed framework—single/multi-source black-box domain adaption for sensor time series data.

maximize the certainty of making task-specific predictions.
Guo et al. [27] constructed a point-to-set metric to cap-
ture the relationship between the target domain and different
source domains. These multisource domain adaption methods
are all applied in computer vision and natural language pro-
cessing. CALDA [28] transfers the distributions of weights
in multisource time series domain adaptation via contrastive
adversarial learning. CoDATS [10] considers weak supervision
of the target domain’s label proportion to design a convo-
lution deep domain adaptation model for time series data.
MRTN [29] designs a multisource-domain-refined adversarial
adaptation network that is applied in fault diagnosis task under
both domain and category inconsistencies. The above methods
depend on the availability of labeled data in the source sensor
time series. Moreover, these methods may be inefficient and
time-consuming when a source-domain time series model has
ample parameter space.
Source-Free Domain Adaption: Source-free domain adap-
tion has been proposed to solve the data privacy and secu-
rity issues in domain adaption. Source-free approaches are
classified into two types: 1) self-supervision and 2) virtual
source transfer. The self-supervision methods adopt pseudo-
label-based strategy by combining information maximization
and self-supervised pseudo-labeling. SHOT [30] develops a
target-specific feature extraction module and implicitly aligns
representations of the target domain with the source model.
The virtual source transfer methods adopt generative adversar-
ial nets (GANs) to generate more samples. MA [31] applies
a class conditional generative adversarial network to gener-
ate target-style data. However, because the model weights are
exposed, there is a possibility of data leaking with source-free
domain adaption.
Black-Box Domain Adaption: There is little research on
black-box domain adaption. DINE [32] introduces a new two-
step knowledge adaption framework called DIstill and fine-
tuNE to solve the black-box domain adaption. B2UDA [33]
adopts iterative learning to conduct noisy labeling and to learn
with noisy labels. However, on the one hand, these methods
are used for image data, which is inappropriate for time series
data. On the other hand, these methods apply a simple mean
ensemble method for multisource black-box domain adaption,
which does not take negative transfer into consideration.
Federated Domain Adaption: Federated learning [34] is a
distributed machine learning technique that learns a shared
prediction model across multiple decentralized edge devices
or servers holding local data samples, without exchanging
them. FADA [35] applies adversarial training to align the rep-
resentations learned among the different nodes with the data
distribution of the target node. FedKA [36] provides a fed-
erated multisource domain adaptation method to improve the
model’s generality in a target domain. However, the communi-
cation cost restricts the application of federated learning. Also,
privacy leakage is still a concern for the enterprise.
B. Teacher–Student Learning
Teacher–student learning trains a student model under
the supervision of the true labels and the teacher model’s
guidance [37]. Teacher–student learning has been applied
in image classification [38] and emotion recognition [39].
Traditional teacher–student learning is employed to build a
smaller model with faster inference speed and maintain the
model accuracy. Recently, teacher–student learning in neural
machine translation [40], speech recognition [41], and image

Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on December 07,2023 at 14:44:18 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
4 IEEE TRANSACTIONS ON CYBERNETICS
classification [42] has also verified the effectiveness in domain
adaption tasks. However, on the one hand, because the data
could contain sensitive information, thus privacy and secu-
rity are the main concerns. On the other hand, the validity of
teacher–student learning in black-box domain adaption has not
been explored.
III. S INGLE /M ULTI -S OURCE B LACK -B OX D OMAIN
A DAPTION FOR T IME S ERIES DATA
Problem Formulation: Let {θsk }M k=1 be set of source mod-
els, where the kth source model θsk corresponding to the
source domain DkS is learned using the source domain dataset
Nk
{(xik , yki )}i=1 , where xik and yki denote the ith source time
series and the label, respectively, and Nk means the size of
source domain dataset. Also the target domain DT has Nt
unlabeled samples {xit }N t t
i=1 , where xi denotes the target time
series and Nt means the size of target domain dataset. The
UDA attempts to learn a model θt to minimize the target error
Ez∼θT [pt (y|z) − p(y|z, f )], where pt (y|z) is the conditional dis-
tribution of the target domain and p(y|z, f ) is the conditional
distribution of the prediction from the target domain. For the
single-source UDA scenario, the number of source domains
M is equal to 1. Different from standard UDA, black-box
domain adaption only provide a black-box predictor. Both
source-domain data and the parameters of the source model
are all not accessed.
A. Teacher Student Learning for Black-Box Domain
Adaption
The framework of the single/multi-source domain adaption
for sensor time series data is shown in Fig. 2. For UDA for
sensor time series data, the teacher models {θsk }M k=1 which have
been trained on respective source domains, are directly used to
conduct inference on unlabeled target domain time-series data
{xit }Nt
i=1 . Since the soft labels have richer information compared
to the hard labels [43], the soft pseudo labels produced by the
teacher model are harnessed to train the student model. By
designing the loss between the prediction of the student model
and the soft labels of the teacher model, the student model
can not only learn the knowledge transferred from the
teacher model but also obtain specific information from
the unlabeled target domain. In black-box domain adaption,
the teacher–student learning calculates the loss as follows:
 
ŷi = p̂ xit , θs (1)
1 
Nt
  
Lt = Dkl ŷi , p̃ xit , θt
Nt
i=1
where Lt denotes the Kullback–Leibler (KL) divergence loss.
Given an unlabeled time series, the teacher–student learn-
ing loss is defined as the KL divergence between the stu-
dent model’s output probability distributions and the teacher
model’s soft label. When inferencing with the target time
series, the trained student model is used to predict the probabil-
ity distribution of the label. Then argmax function is applied
to get the predicted label with the highest probability. Our
research adopts teacher–student learning for black-box domain
Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on December 07,2023 at 14:44:18 UTC from IEEE Xplore. Restrictions apply.
adaption. We want to train a unified model from various
teacher models or their ensembles for more accurate time
series prediction rather than just compressing the model size.
Teacher–student learning in domain adaption [40], [41], [42]
assumes that source model and source data are accessible. And
we apply teacher–student learning in black-box time series
domain adaption, where the source data cannot be directly
accessible.
B. Temporal Consistency Loss
The outputs of the source models for target samples are
not accurate, sometimes not reliable, and even incomplete.
Specifically, the soft label from the teacher model may lose
important time series information, such as data structure and
trend information. Besides, there is a domain shift between
source domains and target domains [44]. The soft labels from
the output of the teacher model may be noisy that is harmful
to student model training.
Considering the sensor time series data properties, we fol-
low the smoothness assumption proposed in [45] and [46]. The
smoothness assumption encourages the same output when tak-
ing original samples and the perturbed samples within a small
neighborhood as the model’s input and helps to alleviate the
over-fitting. We apply the time-series mask strategies to create
corrupted samples as follows:
x̃i = Mi  xi + (E − Mi )  xi (3)
where xi and xi represent the augmented and original time-
series, respectively.  indicates the elementwise product oper-
ator. Mi ∈ [0, 1]d×r×c is a binary mask matrix and E := 1d×r×c
is all-ones matrix, where d is the number of mask trans-
formations, r is the number of the time steps, and c is the
number of features of xi . We have Mi = [m1 , . . . , md ]T , where
mj ∈ [0, 1]r×c is sampled from a Bernoulli distribution with
probability pm . We adjust the level of difficulty of the model
training by changing the hyperparameter pm to control the
proportion of masked time steps. Traditional mask method
generates a binary mask vector that may set 0 for masked
samples [47], [48]. These methods may destroy the intrin-
sic temporal structure for sensor time series data. However,
the temporal dependency is important for sensor time series.
They fail to explicitly model the temporal dependency when
exploring the mask methods for sensor time series data. We
apply the strategy by adding noise of Gaussian and Gaussian
blur kernel to maintain the temporal structure [49]. Blurring
can extract the trend information of a sensor time series and
(2) adding noise contributes to enhancing the local information of
a sensor time series. So a noisy sample xi in x is generated as

x + σ1 , noise
xi = i (4)
gσ2 (xi ), blur
where σ1 is a Gaussian noise with the mean μ and variance
σ12 and gσ2 is a Gaussian blur kernel with the isotropic vari-
ance σ22 . Our generated time series can not only include the
input space in the target domain but also cover the tempo-
ral information. After generating the augmented samples x̃
from x̃, We use the consistency regularization to construct
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

REN AND CHENG: SINGLE/MULTI-SOURCE BLACK-BOX DOMAIN ADAPTION FOR SENSOR TIME SERIES DATA 5

the loss for the unlabeled samples. The object of consistency

regularization is given as follows:

1 
Nt 
d     
Lu = t
Dkl f x̃i,j , f xit (5)
Nt d
i=1 j=1
t is the jth augmented samples from the target domain
where x̃i,j
sample xit . We denote Nt as the number of the original samples,
d as the number of augmented samples from the ith original
sample and f is the function of student model for black-box
domain adaption.
To alleviate the domain shift and class imbalance issues,
inspired by [50], we develop pseudo labeling with dynamic
thresholds to mine pseudo labels with high confidence for
target data. We use a curriculum learning method to adjust Fig. 3. Confidence score of coalition in multidomain adaption for sensor
thresholds flexibly for different classes to select more confi- time series data.
dence samples. The adjusted thresholds Tt (x) are defined as
follows:

N where α is a hyperparameter empirically set to 1 which is
   
σt (x) = 1 max pm,t (y | un ) > τ applied to control the importance of Lu in black-box domain
n=1 adaption.
  
·1 arg max pm,t (y | un ) = x (6)
σt (x) C. Shapley-Enhanced Method in Multisource Black-Box
βt (x) = (7)
maxx σt Domain Adaption
Tt (x) = βt (x) · τ (8)
The core of multisource black-box domain adaption lies in
where x is the name of one class. Tt (x) is the threshold for a model aggregation scheme. We obtain a set of weights wk
class x at training step t. For example, in the har dataset, corresponding to each of the source models. These weights
Tt (x = “biking”) means the threshold of class “biking” at represent a probability mass function over the source domains,
time step t. τ denotes the predefined threshold of the target with a higher value implying high-transfer ability from that
samples. We set τ to exceed 0.8 to filter noisy labels and particular domain. After weighing the black-box predictors
remain high-quality samples. N is the total number of target with wk , the object of the multisource domain adaption
domain data. In (7), when x is best-learned class, we can find converts to the single-source domain adaption like (10).
σt (x) = maxx σt and βt (x) = 1. For classes are hard to learn, Considering the multisource black-box domain adaption, it
we have βt (x) < 1 and Tt (x) < τt . So more training sam- is essential to assign different weights for the different teacher
ples in these classes are added into the model. We design the models. There are usually two ways to tackle this problem.
enhanced KL divergence loss Lt as 1) arg maxk wk : The method uses the most confident
prediction (maximum probability prediction) among wk .
1 
Nt
      2) wk = (1/K): The method computes the mean of multiple
Lt = 1 max ŷi > Tt arg max ŷi source predictors and uses the mean value as the pseudo
Nt
i=1
   label for target samples.
· Dkl ŷi , p̃ xit , θt . (9) However, because of the model bias and domain shift, the
In this way, we select high-confident samples using Tt to outputs are sometimes assigned to the wrong class when we
cover the information from the teacher models. We apply the input the same sample to the different source predictors. So
curriculum learning strategy [51] to make the value of dynamic these methods may lose important information from the multi-
threshold Tt gradually decrease with the process of model source domains and also do not consider the negative transfer
training with the step t. Similar to curriculum learning, we problems.
train the model in black-box domain adaption from easy sam- Inspired by the game theory [52], [53], we propose the
ples (high-threshold samples) to hard samples (low-threshold shapley-enhanced method to calculate the relevance between
samples). High-threshold samples in the target domain usu- different source domains and target domains. When the out-
ally are more similar to source domains in distribution, while put class probability of the network exceeds a threshold τ , we
low-threshold samples are hard to learn but may have specific assume this predicted label is a highly confident label. The
time-series information. value of a confidence number means the number of confidence
Combining these objectives introduced in (9) and (5), labels. The value of a confidence degree means the confidence
we train the predictive model by minimizing the following degree of these confidence labels. Generally speaking, when
objective function: a source domain is more notable and more relative to the tar-
get domain, both the confidence number and the confidence
Lfinal = Lt + α Lu (10) degree are larger. We further divide these high-confidence

Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on December 07,2023 at 14:44:18 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

6 IEEE TRANSACTIONS ON CYBERNETICS

labels into two types, approving and opposing confidence Algorithm 1: Procedure of Single/Multi Black-Box
labels, to eliminate the negative domain adaption. Domain Adaption for Sensor Time Series Data
We assuming that there are multisource domains denoted N
Input: Black-box source model θsk ;Target data xit i=1
t
;
by S = {DkS }K k=1 . And based on each coalition of source Output:
domains S , S ⊆ S, we want to estimate the confidence score
Target Model θt ;
obtained from S . For each sample xit ∈ DT , we get four val-
Teacher Model Output;
ues (p+ (S ), np+ (S ), p− (S ), np− (S )) from S . As shown in
1
if K > 1 then
Fig. 3, S = {D1S , D2S , D3S , D4S } and we have p+  2
i (S ) = 0.92, Calculate the weight for different teacher models via
−    3
pi (S ) = 0.95, np+ (S ) = 2 and np− (S ) = 1. Then we can
i i Eq. (14);
compute the confidence score for each coalition S of source 4 Apply the model aggregation via Eq. (15);
domains as follows: 5 else
       6 Obtain the pseudo label of teacher model via Eq. (1);
F S = np+ S · max p+ i S 7 end
i
xit ∈DT Student Model Training;
    8
− λ ∗ np− S · max p−
i S (11) 9 for e=1 to Tm do
i
10 for i=1 to maximum iteration do
where xit is the ith samples in target domain. The confidence 11 Acquire a batch time series from the target
score is proposed based on three assumptions. First, when a domain;
coalition of source domains S has a larger maximum prob- 12 Generate augmented time series using Eq. (3);
ability max p+ 
i (S ) which means that the domain has lower
13 Update student output by minimizing the
entropy, the prediction has more information and is high-level objective in Eq. (10);
confidence. Second, most source domains support particular 14 end
label with higher confidence, the prediction is more likely to 15 for c=1 to C do
become the true label. So the confidence number np+ (S ) is 16 Determine the flexible threshold T(c) for class c;
i
higher. Third, if a domain is different from other domains, it 17 end
is possible to cause negative transfer. The hyperparameter λ 18 Update the teacher output via Eq. (16);
controls the penalty degree for the negative transfer. 19 end
Shapley Value: A set function v : 2N → R is required as
input for the Shapley value [52]. Each player i ∈ N receives
attributions si from the Shapley value that sum to v(N). The the weight wk
Shapley value of a player i is determined by  
G Dks
 |S|! ∗ (|N| − |S| − 1)! wk =  . (14)
si = (v(S ∪ i) − v(S)) (12) K
k=1 G Ds
k
N!
S⊆N\i
Our purpose is to aggregate a series of source domain time
we first order the players at random, then we add one player series and set weights according to their relevance to the target
at a time in order, finally we assign each player i its expected domain time series. The formula is shown as follows:
marginal contribution v(S ∪ i) − v(S). S is the set of players 
K  
that come before i in the ordering. In this article, we extend ŷi = wk · p̂ xit , θsk (15)
the Shapley value to multisource domain adaption. To get the k=1
confidence score of the source domain Dks , we need to list
all coalitions of source domains S ⊆ S in (11). We get the where θsk represents the output probability distribution of the
information gain to quantify the contribution of each source model and wk is the weight concerning the teacher model θsk .
domain as follows: wk represents the degree of transfer ability from the particular
     domain k, which is learned according to the unlabeled target
   Sl ! K − Sl  − 1 ! domain data. Assuming that there are n source domains, we
G Ds =
k
have K
K! k=1 wk = 1, wk ≥ 0. To further alleviate the noise in the
Sl ⊆S\{Dks } teacher prediction, we adopt an exponential moving average
    
· F Sl ∪ Dks − F Sl (13) strategy
 
ŷi ← wŷi + (1 − w)p̂ xit , θt (16)
where K is the number of multisource domains. For example,
there are four source domains. Then we get K = 4 and K! = where θt represents the model of the current epoch. We
4! = 4 ∗ 3 ∗ 2 ∗ 1 = 24. Sl represents each coalition of another update teacher predictions after every epoch like [54]. w is
source domains S \ {Dks }, Sl ⊆ S \ {Dks }. G(Dks ) describes a momentum hyperparameter to control the degree of tem-
the contribution of domain k to all domains. If one domain poral ensembling. Algorithm 1 summarized all the steps of
has more contributions to all domains, the domain’s weight is single/multi black-box domain adaption for sensor time series
larger. After getting the gains of different domains, we can get data.

Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on December 07,2023 at 14:44:18 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
REN AND CHENG: SINGLE/MULTI-SOURCE BLACK-BOX DOMAIN ADAPTION FOR SENSOR TIME SERIES DATA
TABLE I
D ETAILS OF T HREE S ENSOR T IME S ERIES DATASETS
IV. E XPERIMENTS
The source and target domain samples can be either univari-
ate or multivariate sensor time series. Multivariate sensor is
more complicated. So we select three multivariate time-series
dataset to verify the model’s performance.
A. Dataset
Our proposed framework is evaluated on three public dataset
as shown in Table I, which collect the sensor time series in
IoT scenarios for different tasks. The reason we select these
datasets is that they are mostly widely used in current time-
series domain adaption circumstances [10], [24], [55]. The
three datasets are representative.
1) har [56]: The har dataset contains multichannel sen-
sor time series data for 30 participants performing six tasks:
1) sitting; 2) standing; 3) walking; 4) biking; 5) stair up; and
6) stair down. The data are collected by heterogeneous devices,
including 31 smartphones, four smartwatches, and one tablet.
2) uwave [57]: We use a gesture recognition dataset
(uwave), in which multichannel sensor time series are col-
lected by the accelerometers from 33 participants showing
various hand gestures.
3) wisdm [58]: The wisdm dataset is collected by 36 sub-
jects performing six tasks that are the same as the har dataset.
But the dataset is more challenging because of the class
imbalance problems.
B. Implementation Details
In our experiment, we run our methods four times with dif-
ferent random seeds and then we report the average accuracies.
We adopt the transformer as the main backbone. We use SGD
as the optimizer with momentum (0.9). We set the batch size
to 64. And the initial learning rate is set as 0.1. We use relu as
the activation function. Also, we add batch normalization and
skip connection in the model. To evaluate the model-agnostic
property of the B2TSDA framework, different model architec-
tures, including FCN, transformer, XceptionTime, XCMPlus,
GRU, and ResCNN, are adopted and evaluated.
C. Baseline Methods
There is little work on time series domain adaption. To
demonstrate the feasibility of the proposed method in this arti-
cle, several representatives and classic baselines in computer
Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on December 07,2023 at 14:44:18 UTC from IEEE Xplore. Restrictions apply.
7
vision are used to compare the performance with our method.
A brief introduction of the mentioned baselines is as follows.
1) MT: The method is known as “source only” in this field
that infers the class directly from the predictions. So it
is a direct model transfer method.
2) HT [59]: The method is applied to train the student
model with one-hot labeling vectors (hard label).
3) ST [59]: The method is applied to train the student
model with probability labeling vectors (soft label).
4) SP-UDA [60]: A source-protected UDA method using a
generative model.
5) IterLNL [33]: The iterative learning method with noisy
labels mixing up the noise rate and categorywise sam-
pling to tackle the unbalanced label problems.
6) CMSS [61]: CMSS is a source selection method using
the curriculum manager.
7) DAEL [62]: DAEL is a knowledge ensemble-based
method, that adopts the domain of adaptive ensemble
learning.
8) SimKD3A [63]: The KD3A is a privacy-preserving unsu-
pervised multisource domain adaption method through
knowledge distillation.
9) DINE [32]: DINE is a two-step knowledge adaptation
framework that combines distillation and fine-tunes to
solve the domain adaption for image data.
For fully comparison, we also compare our method with
the latest source-free domain adaption methods, including
SHOT [64], A2Net [65], GSFDA [66], and DECISION [67].
D. Comparison Results
1) Single-Source: We first present the result of cross-
domain time-series classification on har in Table II, uwave in
Table III and wisdm in Table IV. These three tables describe
the target classification accuracy (source → target) on 14
randomly chosen problems for each dataset, which achieve
domain adaption between users. For example, 0 → in Table II
represents adapting from user 0 to user 2 on the har dataset.
To compare with other black-box methods, we adopt the same
backbone-transformer [68]. It covers the advantage of 1D-
CNN and the attention model. We can see B2TSDA achieves
the best-mean accuracy at 76.4% on har, 74.6% on uwave, and
70.9% on wisdom, which outperforms the strongest black-box
baselines by 0.4% on har, 0.7% on uwave and 1.9% on wisdm
in terms of accuracy. We also test the results using different
backbones to verify that B2TSDA is a model-agnostic method
in ablation study.
2) Multisource: We also study the performance of multi-
source domain black-box time-series domain adaption. The
results on har, uwave and wisdm are, respectively, shown in
Tables V–VII. → 1 in Table V indicates that we select the
data of user 1 as the target domain and other users as the
source domain to achieve multisource domain adaption. We
find that B2TSDA gets competitive results on all three datasets.
KD3A uses the size of the source domains, but our circum-
stances do not provide the information. The KD3A we applied
is a simplified version. We note that B2TSDA in multisource
black-box domain adaption achieves a better result. On wisdm,
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

8 IEEE TRANSACTIONS ON CYBERNETICS

TABLE II
S INGLE -S OURCE D OMAIN A DAPTION P ERFORMANCE ON THE HAR DATASET

TABLE III
S INGLE -S OURCE D OMAIN A DAPTION P ERFORMANCE ON THE UWAVE DATASET

TABLE IV
S INGLE -S OURCE D OMAIN A DAPTION P ERFORMANCE ON THE WISDM DATASET

B2TSDA achieved 78% classification accuracy, outperforming
SimK3DA (76.4%).
E. Ablation Study
We adopt an ablation study to demonstrate the effectiveness
of our designed components, including mask augmentation,
dynamic threshold, and information gain.
1) B2TSDA(w/o TM), which denotes that we adapt the
model without the Temporal Mask, only optimize the
objective Lt in (10).
2) B2TSDA(w/o DT), which eliminates the process of using
the Dynamic Threshold Tt to select confidence labels.
3) B2TSDA(w/o SE), which shows the model without
Shapley-Enhanced method and directly averages the
predictions of multisource models. So the weight of
source models wk are equal.
The effectiveness of each component in our model via abla-
tion studies on har, uwave and wisdm datasets, is shown
in Table VIII, where sing-source domain adaption is on the
top, and multisource domain adaption is on the bottom. With
TM or DT ablated, the performance degrades 2.1%, 1.2%
in single-source domain adaption, and 0.6%, 0.5% in mul-
tisource domain adaption, respectively. The absence of SE
causes 1.7% performance drop in multisource domain adap-
tion. The results demonstrate the effectiveness and rationality
of all design choices in our framework.
F. Model Agnostic Analysis
We also compare the different backbones to verify
the model-agnostic model. We do investigations for dif-
ferent student models in our black-box domain adaption

Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on December 07,2023 at 14:44:18 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

REN AND CHENG: SINGLE/MULTI-SOURCE BLACK-BOX DOMAIN ADAPTION FOR SENSOR TIME SERIES DATA 9

TABLE V
M ULTISOURCE D OMAIN A DAPTION P ERFORMANCE ON THE HAR DATASET

TABLE VI
M ULTISOURCE D OMAIN A DAPTION P ERFORMANCE ON THE UWAVE DATASET

TABLE VII
M ULTISOURCE D OMAIN A DAPTION P ERFORMANCE ON THE WISDM DATASET

TABLE VIII
A BLATION R ESULTS OF D IFFERENT VARIANTS FOR S INGLE -S OURCE
AND M ULTISOURCE B LACK -B OX D OMAIN A DAPTIONS

Fig. 4. Illustration of classification accuracy when training on various

amounts of target domain data on tasks 1→5 on the har dataset and 3→8 on
the uwave dataset based on B2TSDA. (a) task1→task5. (b) task3→task8.

applied as the teacher–student model. As shown in Fig. 4(a)

task. We choose popular time-series models as the back-
bones of the student models, including FCN [69], trans-
former [68], XceptionTime [70], XCMPlus [71], GRU [72],
and ResCNN [73]. XceptionTime [70] applies depthwise sep-
arable convolutions, adaptive average pooling, and a novel
no-linear normalization technique in temporal and spatial time-
series. We use the transformer [68] encoder to extract the
importance of time-series representation. XCMPlus [71] is an
explainable CNN for time series classification, Classic deep
learning models for time series, FCN [69], GRU [72], is also
and (b), we can see that sometimes a simple backbone is more
accurate than the complex model. That is to say, we can choose
a simple backbone as a student model. Since a simple model
is effective, it is unnecessary to choose the complex model as
the same as the teacher model.
G. Robustness To Negative Transfer
Irrelevant and malicious source domains in har dataset are
constructed to verify the effectiveness of sharply enhanced
methods and robustness to negative transfer. We select domain

Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on December 07,2023 at 14:44:18 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
10
TABLE IX
M ULTISOURCE D OMAIN A DAPTION P ERFORMANCE W ITH I RRELEVANT
AND M ALICIOUS D OMAINS
TABLE X
M ULTISOURCE D OMAIN A DAPTION W EIGHTS A SSIGNED TO THE
I RRELEVANT AND M ALICIOUS D OMAINS
Fig. 5. Illustration of classification accuracy when training under different
parameters based on B2TSDA. (a) Sensitivity to α. (b) Sensitivity to λ.
5 in har dataset as the target domain. To construct an
irrelevant domain, we select domain 2 as the irrelevant domain,
denoted by IR-2. Because the domain 2 is different from
other domains. To construct a malicious domain, we con-
duct a poisoning attack on the high-quality domain-domain
4 with 30% wrong labels, denoted by AT-30. Also, we report
the performance with the bad domain dropped. We compare
the shapley-enhanced methods with two common strategies:
1) mean focus and 2) consensus focus in SimKD3A [63]. From
the results shown in Tables IX and X, we can get the following
information.
1) Since the domain-drop outperforms the other methods,
the negative transfer for IR-2 truly exists.
2) The consensus focus method and shapley-enhanced
method are robust to the irrelevant domains and mali-
cious domains, because the weights for the irrelevant
domains and malicious domains are low.
3) Shapley-enhanced method performs better in malicious
domains than other techniques because it gives the
malicious domain lower weight (5.1% for AT-30).
H. Parameter Sensitivity
In this section, we evaluate the sensitivity of two hyperpa-
rameters: 1) α controlling the importance of Lu on tasks 1→5
on the har dataset and 2) λ controlling the penalty degree for
the negative transfer on multisource domain adaption →1 on
the uwave dataset. From results of hyperparameters in Fig. 5,
we find that the best parameter of α is around 0.6 and the
best parameter of λ is around 0.4. Different values have major
influences on the model performances and need to be tuned
according to the industrial task.
Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on December 07,2023 at 14:44:18 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON CYBERNETICS
V. C ONCLUSION
In this article, we propose a black-box domain adaption
framework for time series data (B2TSDA). We design the con-
sistency loss by combining dynamic threshold and adaptive
mask to balance the learning difficulties of different classes
and maintain the consistency of temporal information. For the
multisource black-box domain adaption, we further propose
a Shapley-enhanced method to weigh the information from
different teacher models. Experiment results on both single-
source domain adaption and multisource domain adaption
show that our framework has superior performance compared
to other black-box UDA methods. There are some exciting
directions to pursue, including: we suspect that our time-series
mask algorithms can perform well in the universal UDA tasks;
when there are too many source models to utilize, it would be
interesting to study how to improve computational efficiency.
R EFERENCES
[1] J. Roche, V. De-Silva, J. Hook, M. Moencks, and A. Kondoz, “A
multimodal data processing system for LiDAR-based human activity
recognition,” IEEE Trans. Cybern., vol. 52, no. 10, pp. 10027–10040,
Oct. 2022.
[2] L. Vu, V. L. Cao, Q. U. Nguyen, D. N. Nguyen, D. T. Hoang, and
E. Dutkiewicz, “Learning latent representation for IoT anomaly detec-
tion,” IEEE Trans. Cybern., vol. 52, no. 5, pp. 3769–3782, May 2022.
[3] M. Liu, J. Zhang, C. Lian, and D. Shen, “Weakly supervised deep
learning for brain disease prognosis using MRI and incomplete clin-
ical scores,” IEEE Trans. Cybern., vol. 50, no. 7, pp. 3381–3392,
Jul. 2020.
[4] Z. Chen et al., “A multi-source weighted deep transfer network for open-
set fault diagnosis of rotary machinery,” IEEE Trans. Cybern., vol. 53,
no. 3, pp. 1982–1993, Mar. 2023.
[5] C. Liu, K. Wang, Y. Wang, and X. Yuan, “Learning deep multimanifold
structure feature representation for quality prediction with an industrial
application,” IEEE Trans. Ind. Informat., vol. 18, no. 9, pp. 5849–5858,
Sep. 2022.
[6] Y. Ganin et al., “Domain-adversarial training of neural
networks,” J. Mach. Learn. Res., vol. 17, no. 1, pp. 2096–2030,
2016.
[7] H. I. Fawaz, G. Forestier, J. Weber, L. Idoumghar, and P.-A. Muller,
“Transfer learning for time series classification,” in Proc. IEEE Int. Conf.
Big Data (Big Data), 2018, pp. 1367–1376.
[8] Y. Feng, J. Chen, S. He, T. Pan, and Z. Zhou, “Globally localized
multisource domain adaptation for cross-domain fault diagnosis with
category shift,” IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 6,
pp. 3082–3096, Jun. 2023.
[9] S. Liu, H. Zhang, and Y. Jin, “A survey on computationally effi-
cient neural architecture search,” J. Autom. Intell., vol. 1, no. 1, 2022,
Art. no. 100002.
[10] G. Wilson, J. R. Doppa, and D. J. Cook, “Multi-source deep domain
adaptation with weak supervision for time-series sensor data,” in
Proc. 26th ACM SIGKDD Int. Conf. Knowl. Disc. Data Min., 2020,
pp. 1768–1778.
[11] G. Cai, L. He, M. Zhou, H. Alhumade, and D. Hu, “Learning smooth
representation for unsupervised domain adaptation,” IEEE Trans. Neural
Netw. Learn. Syst., vol. 34, no. 8, pp. 4181–4195, Aug. 2023.
[12] S. Xie, Z. Zheng, L. Chen, and C. Chen, “Learning semantic represen-
tations for unsupervised domain adaptation,” in Proc. Int. Conf. Mach.
Learn., 2018, pp. 5423–5432.
[13] S. Zhao et al., “A review of single-source deep unsupervised visual
domain adaptation,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33,
no. 2, pp. 473–493, Feb. 2022.
[14] Y. Yao and G. Doretto, “Boosting for transfer learning with multiple
sources,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern
Recognit., 2010, pp. 1855–1862.
[15] D.-H. Lee, “Pseudo-label: The simple and efficient semi-supervised
learning method for deep neural networks,” in Proc. Workshop
Challenges Represent. Learn., vol. 3, 2013, p. 896.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
REN AND CHENG: SINGLE/MULTI-SOURCE BLACK-BOX DOMAIN ADAPTION FOR SENSOR TIME SERIES DATA
[16] B. Sun and K. Saenko, “Deep CORAL: Correlation alignment for
deep domain adaptation,” in Proc. Eur. Conf. Comput. Vis., 2016,
pp. 443–450.
[17] C. Chen et al., “HoMM: Higher-order moment matching for unsuper-
vised domain adaptation,” in Proc. AAAI Conf. Artif. Intell., vol. 34,
2020, pp. 3422–3429.
[18] J. Yang, Y. Xu, H. Cao, H. Zou, and L. Xie, “Deep learning and transfer
learning for device-free human activity recognition: A survey,” J. Autom.
Intell., vol. 1, no. 1, 2022, Art. no. 100007.
[19] K. Bousmalis, G. Trigeorgis, N. Silberman, D. Krishnan, and D. Erhan,
“Domain separation networks,” in Proc. Adv. Neural Inf. Process. Syst.,
vol. 29, 2016, pp. 343–351.
[20] J. Shen, Y. Qu, W. Zhang, and Y. Yu, “Wasserstein distance guided rep-
resentation learning for domain adaptation,” in Proc. 32nd AAAI Conf.
Artif. Intell., 2018.
[21] W. Zhang, W. Ouyang, W. Li, and D. Xu, “Collaborative and adversar-
ial network for unsupervised domain adaptation,” in Proc. IEEE Conf.
Comput. Vis. Pattern Recognit., 2018, pp. 3801–3809.
[22] S. Motiian, Q. Jones, S. Iranmanesh, and G. Doretto, “Few-shot adver-
sarial domain adaptation,” in Proc. Adv. Neural Inf. Process. Syst.,
vol. 30, 2017, pp. 6670–6680.
[23] H. Zhao, Q. Zheng, K. Ma, H. Li, and Y. Zheng, “Deep representation-
based domain adaptation for nonstationary EEG classification,” IEEE
Trans. Neural Netw. Learn. Syst., vol. 32, no. 2, pp. 535–545,
Feb. 2021.
[24] M. Ragab, E. Eldele, Z. Chen, M. Wu, C.-K. Kwoh, and X. Li,
“Self-supervised autoregressive domain adaptation for time series
data,” IEEE Trans. Neural Netw. Learn. Syst., early access, Jun. 23,
2022, doi: 10.1109/TNNLS.2022.3183252.
[25] H. Zhao, S. Zhang, G. Wu, J. M. Moura, J. P. Costeira, and G. J. Gordon,
“Adversarial multiple source domain adaptation,” in Proc. Adv. Neural
Inf. Process. Syst., vol. 31, 2018, pp. 8559–8570.
[26] Q. Xie, Z. Dai, Y. Du, E. Hovy, and G. Neubig, “Controllable invariance
through adversarial feature learning,” 2017, arXiv:1705.11122.
[27] J. Guo, D. Shah, and R. Barzilay, “Multi-source domain adaptation
with mixture of experts,” in Proc. Conf. Empir. Methods Natural Lang.
Process., 2018, pp. 4694–4703.
[28] G. Wilson, J. R. Doppa, and D. J. Cook, “CALDA: Improving
multi-source time series domain adaptation with contrastive adversarial
learning,” 2021, arXiv:2109.14778.
[29] Z. Chai, C. Zhao, and B. Huang, “Multisource-refined transfer network
for industrial fault diagnosis under domain and category inconsisten-
cies,” IEEE Trans. Cybern., vol. 52, no. 9, pp. 9784–9796, Sep. 2022.
[30] J. Liang, D. Hu, and J. Feng, “Do we really need to access the source
data? Source hypothesis transfer for unsupervised domain adaptation,” in
Proc. Int. Conf. Mach. Learn., 2020, pp. 6028–6039.
[31] R. Li, Q. Jiao, W. Cao, H.-S. Wong, and S. Wu, “Model adapta-
tion: Unsupervised domain adaptation without source data,” in Proc.
IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2020, pp. 9641–9650.
[32] J. Liang, D. Hu, J. Feng, and R. He, “DINE: Domain adaptation from
single and multiple black-box predictors,” in Proc. IEEE/CVF Conf.
Comput. Vis. Pattern Recognit., 2022, pp. 1–11.
[33] H. Zhang, Y. Zhang, K. Jia, and L. Zhang, “Unsupervised domain
adaptation of black-box source models,” 2021, arXiv:2101.02839.
[34] J. Konečnỳ, H. B. McMahan, F. X. Yu, P. Richtárik, A. T. Suresh, and
D. Bacon, “Federated learning: Strategies for improving communication
efficiency,” 2016, arXiv:1610.05492.
[35] X. Peng, Z. Huang, Y. Zhu, and K. Saenko, “Federated adversar-
ial domain adaptation,” in Proc. Int. Conf. Learn. Represent., 2019,
pp. 1–19.
[36] Y. Sun, N. Chong, and O. Hideya, “Multi-source domain adaptation
based on federated knowledge alignment,” 2022, arXiv:2203.11635.
[37] H. Chen, Y. Wang, C. Xu, C. Xu, and D. Tao, “Learning student
networks via feature embedding,” IEEE Trans. Neural Netw. Learn.
Syst., vol. 32, no. 1, pp. 25–35, Jan. 2021.
[38] S. Li et al., “Distilling a powerful student model via online knowledge
distillation,” IEEE Trans. Neural Netw. Learn. Syst., early access, Mar. 7,
2022, doi: 10.1109/TNNLS.2022.3152732.
[39] T. Gu, Z. Wang, X. Xu, D. Li, H. Yang, and W. Du, “Frame-level
teacher–student learning with data privacy for EEG emotion recogni-
tion,” IEEE Trans. Neural Netw. Learn. Syst., early access, Apr. 29,
2022, doi: 10.1109/TNNLS.2022.3168935.
[40] X. Tan, Y. Ren, D. He, T. Qin, Z. Zhao, and T.-Y. Liu,
“Multilingual neural machine translation with knowledge distilla-
tion,” 2019, arXiv:1902.10461.
Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on December 07,2023 at 14:44:18 UTC from IEEE Xplore. Restrictions apply.
11
[41] Z. Meng, J. Li, Y. Gaur, and Y. Gong, “Domain adaptation via
teacher–student learning for end-to-end speech recognition,” in Proc.
IEEE Autom. Speech Recognit. Understand. Workshop (ASRU)2019,
pp. 268–275.
[42] S. You, C. Xu, C. Xu, and D. Tao, “Learning from multiple teacher
networks,” in Proc. 23rd ACM SIGKDD Int. Conf. Knowl. Disc. Data
Min., 2017, pp. 1285–1294.
[43] G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural
network,” Comput. Sci., vol. 14, no. 7, pp. 38–39, 2015.
[44] A. Hussein and H. Hajj, “Domain adaptation with representation learning
and nonlinear relation for time series,” ACM Trans. Internet Things,
vol. 3, no. 2, pp. 1–26, 2022.
[45] T. Miyato, S.-I. Maeda, M. Koyama, and S. Ishii, “Virtual adversarial
training: A regularization method for supervised and semi-supervised
learning,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 41, no. 8,
pp. 1979–1993, Aug. 2019.
[46] O. Chapelle, B. Scholkopf, and A. Zien, “Semi-supervised learning
(Chapelle, O. et al., Eds.; 2006) [book reviews],” IEEE Trans. Neural
Netw., vol. 20, no. 3, p. 542, Mar. 2009.
[47] W.-N. Hsu, B. Bolte, Y.-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and
A. Mohamed, “HuBERT: Self-supervised speech representation learning
by masked prediction of hidden units,” IEEE/ACM Trans. Audio, Speech,
Lang. Process., vol. 29, pp. 3451–3460, 2021.
[48] Y. Zhao, M. Elhousni, Z. Zhang, and X. Huang, “Distance trans-
form pooling neural network for LiDAR depth completion,” IEEE
Trans. Neural Netw. Learn. Syst., early access, Dec. 13, 2021,
doi: 10.1109/TNNLS.2021.3129801.
[49] R. J. Hyndman and G. Athanasopoulos, Forecasting: Principles and
Practice. Melbourne, VIC, Australia: OTexts, 2018.
[50] B. Zhang et al., “FlexMatch: Boosting semi-supervised learning with
curriculum pseudo labeling,” in Proc. Adv. Neural Inf. Process. Syst.,
vol. 34, 2021, pp. 1–16.
[51] Y. Bengio, J. Louradour, R. Collobert, and J. Weston, “Curriculum
learning,” in Proc. 26th Annu. Int. Conf. Mach. Learn., 2009, pp. 41–48.
[52] S. Lipovetsky and M. Conklin, “Analysis of regression in game the-
ory approach,” Appl. Stochastic Models Bus. Ind., vol. 17, no. 4,
pp. 319–330, 2001.
[53] S. M. Lundberg and S.-I. Lee, “A unified approach to interpreting model
predictions,” in Proc. Adv. Neural Inf. Process. Syst., vol. 30, 2017,
pp. 4768–4777.
[54] K. Kim, B. Ji, D. Yoon, and S. Hwang, “Self-knowledge distillation
with progressive refinement of targets,” in Proc. IEEE/CVF Int. Conf.
Comput. Vis., 2021, pp. 6567–6576.
[55] Q. Liu and H. Xue, “Adversarial spectral kernel matching for unsuper-
vised time series domain adaptation,” in Proc. Int. Joint Conf. Artif.
Intell., 2021, pp. 2744–2750.
[56] A. Stisen et al., “Smart devices are different: Assessing and mitigating-
mobile sensing heterogeneities for activity recognition,” in Proc. 13th
ACM Conf. Embedded Netw. Sens. Syst., 2015, pp. 127–140.
[57] J. Liu, Z. Wang, L. Zhong, J. Wickramasuriya, and V. Vasudevan,
“uWave: Accelerometer-based personalized gesture recognition and its
applications,” in Proc. IEEE Int. Conf. Pervasive Comput. Commun.,
2009, pp. 1–9.
[58] J. R. Kwapisz, G. M. Weiss, and S. A. Moore, “Activity recognition
using cell phone accelerometers,” ACM SigKDD Explorations Newslett.,
vol. 12, no. 2, pp. 74–82, 2011.
[59] H. Wang, Q. Zhang, J. Wu, S. Pan, and Y. Chen, “Time series feature
learning with labeled and unlabeled data,” Pattern Recognit., vol. 89,
pp. 55–66, May 2019.
[60] B. Yang, A. J. Ma, and P. C. Yuen, “Revealing task-relevant model mem-
orization for source-protected unsupervised domain adaptation,” IEEE
Trans. Inf. Forensics Security, vol. 17, pp. 716–731, 2022.
[61] L. Yang, Y. Balaji, S.-N. Lim, and A. Shrivastava, “Curriculum manager
for source selection in multi-source domain adaptation,” in Proc. Eur.
Conf. Comput. Vis., 2020, pp. 608–624.
[62] K. Zhou, Y. Yang, Y. Qiao, and T. Xiang, “Domain adaptive ensem-
ble learning,” IEEE Trans. Image Process., vol. 30, pp. 8008–8018,
2021.
[63] H. Feng et al., “KD3A: Unsupervised multi-source decentralized domain
adaptation via knowledge distillation,” in Proc. Int. Conf. Mach. Learn.,
2021, pp. 3274–3283.
[64] J. Liang, D. Hu, Y. Wang, R. He, and J. Feng, “Source data-absent
unsupervised domain adaptation through hypothesis transfer and label-
ing transfer,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 11,
pp. 8602–8617, Nov. 2022.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

12 IEEE TRANSACTIONS ON CYBERNETICS

[65] H. Xia, H. Zhao, and Z. Ding, “Adaptive adversarial network for source- Lei Ren (Member, IEEE) received the Ph.D. degree
free domain adaptation,” in Proc. IEEE/CVF Int. Conf. Comput. Vis., in computer science from the Institute of Software,
2021, pp. 9010–9019. Chinese Academy of Sciences, Beijing, China,
[66] S. Yang, Y. Wang, J. Van De Weijer, L. Herranz, and S. Jui, “Generalized in 2009.
source-free domain adaptation,” in Proc. IEEE/CVF Int. Conf. Comput. He is currently a Professor with the School
Vis., 2021, pp. 8978–8987. of Automation Science and Electrical Engineering,
[67] S. M. Ahmed, D. S. Raychaudhuri, S. Paul, S. Oymak, and Beihang University; Zhongguancun Laboratory;
A. K. Roy-Chowdhury, “Unsupervised multi-source domain adaptation and the State Key Laboratory of Intelligent
without access to source data,” in Proc. IEEE/CVF Conf. Comput. Vis. Manufacturing System Technology, Beijing. His
Pattern Recognit., 2021, pp. 10103–10112. research interests include neural networks and
[68] G. Zerveas, S. Jayaraman, D. Patel, A. Bhamidipaty, and C. Eickhoff, “A deep learning, industrial digital twin, and industrial
transformer-based framework for multivariate time series representation AI applications.
learning,” in Proc. 27th ACM SIGKDD Conf. Knowl. Disc. Data Min., Dr. Ren serves as an Associate Editor for the IEEE T RANSACTIONS
2021, pp. 2114–2124. ON N EURAL N ETWORKS AND L EARNING S YSTEMS and other international
[69] Z. Wang, W. Yan, and T. Oates, “Time series classification from scratch journals.
with deep neural networks: A strong baseline,” in Proc. Int. Joint Conf.
Neural Netw. (IJCNN), 2017, pp. 1578–1585.
[70] E. Rahimian, S. Zabihi, S. F. Atashzar, A. Asif, and A. Mohammadi,
“Xceptiontime: Independent time-window xceptiontime architecture for
hand gesture classification,” in Proc. IEEE Int. Conf. Acoust., Speech
Signal Process. (ICASSP) 2020, pp. 1304–1308.
[71] K. Fauvel, T. Lin, V. Masson, É. Fromont, and A. Termier, “XCM: An
Xuejun Cheng (Graduate Student Member, IEEE)
explainable convolutional neural network for multivariate time series
received the bachelor’s and master’s degrees in
classification,” Mathematics, vol. 9, no. 23, p. 3137, 2021.
automation from Beihang University, Beijing, China,
[72] L. Ren, X. Cheng, X. Wang, J. Cui, and L. Zhang, “Multi-scale
in 2016 and 2019, respectively, where he is cur-
dense gate recurrent unit networks for bearing remaining useful
rently pursuing the Ph.D. degree with the School
life prediction,” Future Gener. Comput. Syst., vol. 94, pp. 601–609,
of Automation Science and Electrical Engineering.
May 2019.
His research interests include deep learning for
[73] X. Zou, Z. Wang, Q. Li, and W. Sheng, “Integration of residual network
time series forecasting and deep learning for Internet
and convolutional neural network along with various activation func-
of Things.
tions and global pooling for time series classification,” Neurocomputing,
vol. 367, pp. 39–45, Nov. 2019.

Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on December 07,2023 at 14:44:18 UTC from IEEE Xplore. Restrictions apply.