GPS Solutions (2021) 25:25

https://doi.org/10.1007/s10291-020-01074-y

ORIGINAL ARTICLE

Joint time–frequency mask and convolutional neural network

for real‑time separation of multipath in GNSS deformation monitoring
Yuan Tao1 · Chao Liu1,2,3   · Chunyang Liu1,3 · Xingwang Zhao1 · Haojie Hu1 · Haiqiang Xin4

Received: 29 October 2020 / Accepted: 16 December 2020 / Published online: 3 January 2021
© The Author(s), under exclusive licence to Springer-Verlag GmbH, DE part of Springer Nature 2021

Abstract
As one of the leading technologies of real-time dynamic deformation monitoring, global navigation satellite system (GNSS)
has been widely used in deformation monitoring. Multipath error is the primary error source that limits the application of
GNSS high-precision deformation monitoring, which cannot be eliminated by the double-differenced technique. It is a prob-
lem to separate and mitigate multipath in the GNSS deformation monitoring series in real-time. Therefore, we propose an
approach to solve this problem, named time–frequency mask and convolutional neural network (TFM–CNN). The specific
processes are as follows: (1) TFM–CNN network construction. We add a full-band deformation to the multipath and obtain
the spectrogram of the signal by the short-time Fourier transform (STFT); meanwhile, the ideal ratio mask (IRM) is used to
obtain the corresponding time–frequency mask matrix based on the spectrogram; furthermore, one-dimensional CNN mines
the mapping relationship between the spectrogram and the time–frequency mask matrix. (2) Multipath separation. We obtain
the spectrogram of the GNSS real-time deformation monitoring series by STFT. Then, we estimate its time–frequency mask
matrix by the network. The matrix is multiplied by the spectrogram of the monitoring series to obtain the spectrogram of the
multipath. Finally, we perform the inverse STFT to obtain the multipath in the GNSS monitoring series. The experimental
results show that by training GNSS data in only a specific direction (such as the north direction), the mapping relationship
between the spectrogram of multipath and the time–frequency mask matrix can be obtained, which can effectively separate
multipath of the GNSS monitoring series in different observation environment in real-time. TFM-CNN significantly improves
the monitoring accuracy and achieves millimeter level dynamic deformation monitoring.

Keywords  Time–frequency mask · Convolutional neural network · Deformation monitoring · Multipath mitigation · GNSS

Introduction monitoring. However, many error terms limit GNSS

Global navigation satellite system (GNSS) has the advan-
tages of real-time, high-precision and automation. It
has become an essential technology for structural health
* Chao Liu
chaoliu0202@gmail.com
1
School of Spatial Information and Geomatics Engineering,
Anhui University of Science and Technology,
Huainan 232001, China
2
School of Mining and Geomatics, Hebei University
of Engineering, Handan 056038, China
3
Jiangsu Key Laboratory of Resources and Environmental
Information Engineering, China University of Mining
and Technology, Xuzhou 221116, China
4
Xinjiang Academy of Surveying and Mapping,
Urumqi 830002, China
high-precision positioning services. Although the double-
differenced technique for GNSS short baseline can elimi-
nate satellite clock error, receiver clock error and satellite
ephemeris error and significantly weaken the influence of
geometry-related errors, such as tropospheric delay error
and ionospheric delay error, the multipath cannot be elimi-
nated. Therefore, the multipath has become the primary
error source affecting the high-precision GNSS applications.
Scholars have carried out much research to mitigate mul-
tipath or mitigate its influence on positioning accuracy. The
efforts are made from two facets, hardware and software.
The hardware-based approach is to improve the performance
of the antenna of GNSS receiver to reduce multipath, such
as dual-polarized antennas (Groves et al. 2010; Zhang et al.
2017), ground planes (Maqsood et al. 2013) and choke rings
(Danskin et al. 2009). They can shield some refraction sig-
nals to avoid multipath, which has a limited performance

13
Vol.:(0123456789)
25 
Page 2 of 13
to reduce multipath. We assume the GNSS data processing
related methods as a software-based approach. The related
research based on mathematical principles can be further
divided into two types: the stochastic and function mod-
els. The stochastic model endues the observations of dif-
ferent systems and satellites with a reasonable prior vari-
ance–covariance matrix (Brunner et al. 1999; Zimmermann
et al. 2017; Xi et al. 2020). It can weaken the influence of
multipath on the estimated parameters and improve position-
ing accuracy. However, it is necessary to further mitigate
multipath in the function model. The function model takes
multipath as the systematic error and uses multipath as the
parameter (Yuan et al. 2010) to estimate with the coordinate
parameters, or uses multi-antenna strategies (Ray et al. 2001)
to estimate multipath. It is arbitrary to use the multipath as
parameters so that it could affect the structural space of the
GNSS adjustment model. In other words, that will cause the
excessive parameterization of the GNSS adjustment model
so that it could reduce the accuracy of coordinate estimation.
Multipath is the interference signals formed by the refracted
signal and the direct signal, which is affected by factors such
as the position, attributes of the reflector and the frequency
of the signal. Moreover, the essential determinant is the geo-
metric relationship between the satellite, the GNSS antenna
and the reflector. When the position of the GNSS antenna is
unchanged and the environment around the antenna keeps
the same, there is a direct mapping relationship between the
position or operating cycle of the satellite and the multipath.
According to this fact, the sidereal filtering (SF) method is pro-
posed to mitigate multipath, using the repeatability of satellites
and multipath under the assumption that all visible satellites
have the same and unchanged operating period (Genrich and
Bock 1992; Choi et al. 2004). Considering that the operat-
ing periods of the satellites are slightly different (Agnew and
Larson 2007), Zhong et al. (2010) proposed to convert the
double-difference residual series of GPS carrier phase obser-
vations into single-difference residual series based on the "zero
mean" assumption (Alber et al. 2000). They implemented the
advanced sidereal filtering (ASF) on each GPS satellite in the
observation-domain, which further improves the performance
of SF and mitigates the effect of multipath. Ye et al. (2015) and
Zhang et al. (2018) expanded the ASF method and applied it
to mitigate the multipath of three types of satellites in beidou
navigation satellite system (BDS). The SF and ASF methods
can effectively mitigate the multipath on positioning accuracy,
but both require pre-calculation of the satellite operating peri-
ods. However, the operating period of each satellite changes
slowly over time, which reduces the real-time performance of
mitigating multipath. Dong et al. (2016) established a hemi-
spherical multipath map (MHM) using the spatial correlation
of multipath instead of temporal correlation to address this
problem. Moreover, it further used the direct mapping relation-
ship between multipath and satellite position when the GNSS
13
GPS Solutions (2021) 25:25
antenna and reflector did not change significantly. The differ-
ence is that the elevation angle and azimuth angle are used
to describe the satellite position. Therefore, the MHM model
is more feasible and reliable than the SF and ASF methods.
Scholars have improved and expanded the MHM model. For
example, Wang et al. (2019) proposed an improved MHM
method, T-MHM (trend surface analysis-based multipath
hemispherical map), to improve multipath calibration accu-
racy. Moreover, MHM and its extensions have been applied to
different researches such as precise point positioning (Zheng
et al. 2019) and dynamic relative positioning (Cai et al. 2016).
In the practice of deformation monitoring, the position of
the GNSS monitoring station is continually changing, which
is affected by the dynamic deformation of the monitoring
body. SF, MHM and their improved methods are signifi-
cantly challenged in multipath mitigation. Moreover, GNSS
monitoring sites have been continuously deforming and it is
challenging to establish corresponding multipath models for
some deformable bodies in operation. We want to investigate
whether the establishment of multipath models in other loca-
tions can be used to mitigate multipath in real-time for the
GNSS monitoring sites.
Inspired by the speech separation field that it is neces-
sary to separate the target speech from disturbed speech and
noise (Wang et al. 2014; Wang and Chen 2018), we can
treat multipath as the "target speech" and the deformation as
the "disturbed speech". The GNSS deformation monitoring
series is the same as digital signals in speech separation.
Therefore, we assume the speech separation tool can be used
to mitigate multipath in the GNSS deformation monitoring
series in real-time.
Based on the identified methods, we propose an approach
jointing the time–frequency mask and convolutional neural
network (TFM–CNN) to mitigate multipath in real-time.
The purpose of the method is to mine the deep frequency-
domain characteristics of multipath. First, the one-dimen-
sional time-domain information of the multipath signal is
converted into a two-dimensional spectrogram by short-time
Fourier transform (STFT) (Durak and Arikan 2003). Then,
we construct the time–frequency mask matrix corresponding
to the multipath signal and we use one-dimensional CNN
(1D-CNN) to establish the mapping relationship between
the spectrogram and the mask matrix. Finally, we use the
established mapping relationship to separate multipath in
the GNSS deformation monitoring series of other locations
to improve the GNSS dynamic monitoring accuracy.
Proposed method
The framework of the TFM-CNN is displayed in Fig. 1.
First, we apply the time–frequency mask method to GNSS
data processing and explain the primary mechanism of
GPS Solutions (2021) 25:25 Page 3 of 13  25
separating multipath of GNSS deformation monitoring by
constructing a mask matrix.
Then, 1D-CNN mines the deep non-linear relationship
between the spectrogram and the mask matrix. Specifically,
we transform the one-dimensional time-domain series of the
prior multipath and its mixed simulation deformation into
two-dimensional frequency-domain spectrograms by STFT,
respectively. After that we calculate the mask matrix based
on the spectrograms. The matrix and the spectrogram of
multipath mixed with deformation are used as the learning
target of 1D-CNN to obtain the parameter model of mask
matrix that can be used to estimate multipath.
Time–frequency mask
Single-channel speech separation based on deep learning
has made many significant signs of progress (Weninger et al.
2014; Huang et al. 2015; Luo and Mesgarani 2019). Wang
(2005) proposed the time–frequency mask method, which
has become an effective strategy to solve single-channel
speech separation based on deep learning. For the mixed-
signals with multiple speeches, as long as the time–fre-
quency mask of the target speech is established, it can sepa-
rate the target speech in real-time.
Time–frequency mask can reflect the interrelationship
between the different signals in a mixed-signal. By calcu-
lating the mask matrix of the target signal, the frequency-
domain information of the target signal can be extracted.
Similarly, the GNSS series mixed with deformation can be
regarded as a single-channel digital signal. Therefore, this
method can be transplanted to GNSS deformation monitor-
ing to realize GNSS multipath separation.
The main time–frequency mask methods are the ideal
binary mask (IBM) and ideal ratio mask (IRM). IBM is a
binary mask matrix. It will discard entirely or retain the fre-
quency information at the corresponding position. However,
IRM is a mask matrix with values between 0 and 1. It refines
the processing mode of frequency-domain information and
has a better processing effect than IBM (Srinivasan et al.
2006). The matrix between 0 and 1 retains the target signal
Fig. 1  Framework of TFM–CNN
with the same frequency as the interference signal, rather
than completely discarding it. After obtaining the mask
matrix, the spectrogram of the target signal can be separated
from the spectrogram of the mixed-signal by multiplying the
matrix and the spectrogram of the mixed signal. Therefore,
we use IRM to separate the target signal and IRM is defined
as:
( )𝛽 ( )𝛽
S2 (t, f ) SNR(t, f )
IRM(t, f ) = =
S2 (t, f ) + N 2 (t, f ) SNR(t, f ) + 1
(1)
( 2
)
S (t, f )
SNR(t, f ) = 10 ⋅ lg (2)
N 2 (t, f )
where S2 (t, f ) and N 2 (t, f ) represent the energy of target sig-
nal and mixed-signal in the time–frequency unit of frame t
and frequency f; SNR(t, f ) represents the local signal-to-noise
ratio in the time–frequency unit of frame t and frequency f;
𝛽 is expressed as an adjustable scale factor. It is worth men-
tioning that 𝛽 in speech processing field is generally set to
0.5. Nevertheless, the method is applied in the GNSS field
for the first time; suitable parameters need to be tested. After
many experiments, 𝛽 is set to 0.9.
TFM‑CNN network construction
To effectively separate multipath errors from the GNSS
deformation monitoring series in real-time, we use the
time–frequency mask (i.e., IRM) and CNN, named TFM-
CNN. After obtaining the corresponding spectrograms and
the mask matrix, we use CNN to explore the complicated
non-linear relationship between them. As an essential branch
of deep learning, convolutional neural networks are widely
used in image recognition (Zheng et al. 2017), semantic seg-
mentation (Girshick et al. 2014) and time-series prediction
(Pourtaghi and Lotfollahi-Yaghin 2012; Phan et al. 2018).
Since multi-GNSS multipath in each day is not the same in
frequency-domain, a unified mask matrix cannot be estab-
lished. Therefore, CNN can be used to learn the mapping
relationship between spectrograms of multipath and mask
matrix, estimating the mask matrix from the spectrogram
of monitoring series.
Considering the GNSS monitoring series features, we use
1D-CNN to extract useful representations of input data by
establishing multiple filters and mine the complex features
in coordinate series. We describe the building blocks of the
1D-CNN including an input layer, two convolutional layers
with the rectified linear unit (ReLU) (Nair and Hinton 2010)
as a non-linear activation function, a flatten layer, three lay-
ers of dense layer and batch normalization and two layers
of LeakyReLU (Xu et al. 2015) and dropout (Hinton et al.
2012; Srivastava et al. 2014).
13
25 
Page 4 of 13
The activation functions perform non-linear transforma-
tions on the output results of each layer to improve network
expression ability. The last dense layer of activation func-
tion uses sigmoid ( sigmoid(x) = 1∕(1 + e−x ) ). And we add
the dropout between dense layers prevents over-fitting and
enhances the network generalization ability. The function of
the convolutional layer is to automatically extract the series
features of each frame, which can share weights and reduce
the parameters of the model (LeCun et al. 1998). Specifi-
cally, the input series is convolved with the convolution ker-
nel of each channel and the output feature vector is obtained
based on activation function. The calculation equation is:
⎛� ⎞ to the 1D-CNN model for mining deep frequency-domain
xjl =f ⎜ l−1 l l⎟
x ⋅ kij + bj (3)
⎜i∈M i ⎟ features and learning the mapping relationship. Moreover,
⎝ j ⎠ considering the short-term correlation of the GNSS time-
where xjl is the feature vector corresponding to the j-th con-
volution kernel of the l-th convolution layer; f (⋅) means acti-
vation function; Mj is the receptive domain of the current
neuron; kijl is the convolution kernel vector of the j-th chan-
nel of the l-th convolutional layer corresponding to the i-th
input feature vector; blj is the bias of the j-th channel of the
l-th convolutional layer.
Multipath separation
After determining the network structure, setting the train-
ing data are the primary mission of the network learning
stage. To explore the deep characteristics of multipath in
frequency-domain and avoid high-frequency noise interfer-
ing with training data, we denoise the original GNSS data
based on empirical mode decomposition to obtain multipath.
Then, we add simulated deformation into multipath, which
is a full-band vibration signal (The amplitude is 5 mm, the
frequency range is 0.01 Hz to 0.2 Hz and the interval is
0.001 Hz.). After that we obtain the spectrograms of mul-
tipath and its mixed deformation by STFT, respectively.
(The STFT configurations are as follows: window length:
512, window overlap: 256, window function type: "hann".).
Unfortunately, when STFT transforms a one-dimensional
signal into a two-dimensional spectrogram, the end of the
Fig. 2  Spectrograms of
multipath (left) and its mixed
deformation (middle) and their
time–frequency mask matrix
(right)
13
GPS Solutions (2021) 25:25
signal will be automatically filled with those less than one
frame. ISTFT (inverse short-time Fourier transform) will
distort the end of the signal. Therefore, we extend the origi-
nal series according to the window size of STFT to ensure
that the signal waveform is not affected during ISTFT.
In the network learning stage, the spectrogram of mul-
tipath of mixed deformation is taken as independent vari-
ables of training data, and the mask matrix is taken as
dependent variables of training data, as shown in Fig. 2.
The spectrograms in the left and middle panels display the
signal strength. (The redder the color, the greater the sig-
nal strength.). Then, the two spectrograms are calculated to
obtain the mask matrix in the right panel. We feed the data
series, frame expansion is used to achieve a better learn-
ing effect; that is, the current frame and two frames before
and after are simultaneously trained. By the above training
method, the parameter model is obtained and saved.
The specific operation process of the separation stage is
shown in Fig. 3. First, we obtain the spectrogram of original
data mixed with deformation by STFT. However, when a
one-dimensional series is transformed into a spectrogram,
the phase information of the series is not considered. There-
fore, we need to save the phase information. Then, after the
frame expansion operation is performed on the spectrogram,
the parameter model is loaded to estimate the mask matrix.
Finally, the matrix and the spectrogram are multiplied to
obtain the spectrogram of the separated multipath. After
combining the spectrogram with phase information, ISTFT
is used to obtain multipath.
The original one-dimensional series is transformed into
a two-dimensional spectrogram and the spectrogram enters
the network learning in the order of time–frequency frames,
which reduces the complexity of network model learning.
This strategy not only reduces the parameter dimension of
the network structure but also improves the efficiency of
learning. Besides, we need to determine the stability of net-
work structure through the loss function of validation data.
After extensive testing, the hyperparameters selected in net-
work structure are: the number of convolution kernels is 8
GPS Solutions (2021) 25:25 Page 5 of 13  25

Fig. 3  Specific operation processes of the separation stage

and 16, the convolution kernel sizes are 5 and 7, the dense
layer sizes are 128, 128 and 257, batch size is 32, the dropout
layers are 0.2 and 0.4. and the number of iterations is 500.
The loss function uses MSE (mean square error).

Experiments

In this section, we introduce the data source and the simula-

tion deformation added in the experiment. Then, we focus on
the TFM–CNN method to separate the deformation problem
and carry out experimental demonstrations from multiple
angles. The following methods are the problems that need
experimental demonstrations: Fig. 4  Environment of the stations

Whether the proposed method is feasible for separating

multipath and deformation.
Whether the proposed method is affected by different
training samples.
Whether the proposed method can use training data in a
certain direction to separate deformation in all directions.
Whether the proposed method uses only one training
sample to separate the deformation of different environ-
mental observations.
Data
The experimental data were collected at two GNSS continu-
ously operating reference stations in the campus of Anhui
University of Science and Technology. Figure 4 shows the
environment of the stations. The data selected the original
observation data from day of year (DOY) 286–310 in 2019.
The receivers have a Trimble BD980 board, AT300 antenna.
They can receive GPS, BDS and GLONASS data simultane-
ously with the sampling rate of 1 Hz and the satellite cutoff
angle is 15°. We perform a single epoch calculation on the
combined GPS/BDS/GLONASS observation data to obtain
dynamic coordinate series. Since the baseline distance is
short and there is no difference in the vertical direction,
it can be considered that the tropospheric delay and iono-
spheric delay can be eliminated after the double-differenced
technique. Therefore, there are only multipath and noise in
the multi-GNSS coordinate series.
In our experiments, the data are divided into training data
and test data. The simulated deformation is added to test
data. The frequencies are 0.015, 0.08 and 0.15 Hz and the
corresponding amplitudes of the three groups of vibration
deformation are 5, 10 and 20 mm. Among them, low-fre-
quency deformation and multipath have the same frequency
(Dai et al. 2014).
Feasibility of multi‑GNSS deformation separation
To explore the feasibility of the TFM–CNN method, we ana-
lyze the effect of separating deformation in the east, north
and vertical directions from the time-domain and frequency-
domain. We use the three directions series of DOY286
to train parametric models, respectively, and based on
TFM–CNN for the remaining 24 days of test data to mitigate

13
25 
Page 6 of 13 GPS Solutions (2021) 25:25

Fig. 5  Original series in the

east, north and vertical direc-
tions and their separated mul-
tipath. Note: Original means
original GNSS series, Re-
multipath means the separated
multipath

Fig. 6  Monitoring accuracy and

the improvement before and
after mitigating multipath

Fig. 7  Separated deformation,
simulation deformation and
their residual

multipath. We analyze the feasibility of the method by the
statistical results.
Figure 5 shows the separated multipath and the original
series in the east, north and vertical directions. TFM–CNN
can separate multipath from coordinate series of mixed com-
plex deformation and its amplitude and trend are similar to
the original series. Moreover, the separated multipath have
superior robustness. Figure 6 shows the 24-day statistical
results of deformation monitoring accuracy and improve-
ment before and after mitigating multipath. The east, north
and vertical directions monitoring accuracy of the original
multi-GNSS are 1.3, 1.5 and 4.1 mm. After mitigating mul-
tipath based on TFM–CNN, the monitoring accuracy of east,
north and vertical directions is approximately 0.4, 0.4 and
1.2 mm, respectively. Besides, the 24-day improvement can
reach a minimum of 64%.
To analyze the deformation and noise after mitigating
multipath, we show the separated deformation and simu-
lated deformation for one hour and their residual of one day
in Fig. 7. Although the deformation is affected by noise,
the waveform remains highly consistent with deformation
because the amplitude of the deformation is much larger
than the noise level in the residual. Moreover, the bias of
the residual between deformation and mitigated multipath
series is on the order of ­10–5 mm. We also performed a
statistical analysis on the correlation coefficient, the root

13
GPS Solutions (2021) 25:25 Page 7 of 13  25

mean square (RMS) and bias between deformation and the

series after the 24-day east, north and vertical directions
mitigated multipath, as shown in Fig. 8. For the east and
north directions, the correlation coefficient and RMS can be
maintained at a high level. In the vertical direction, the eval-
uation performance is inferior to other directions because
the noise level in the vertical direction is higher than that
in the east and north directions. For 24 days, the bias of the
east, north and vertical directions can be maintained on the
order of ­10–5 mm. In summary, it can be considered that
TFM-CNN can accurately separate multipath in deforma-
tion monitoring.
In the frequency-domain, the accuracy of the series and
deformation after mitigating multipath can be observed.
Power spectral density (PSD) can reflect the frequency
and power characteristics of the signal. Figure 9 shows the
PSD of each series in the east direction. In the top panel,
we show PSD of three sets of series, which are the original
series, separated multipath and the denoised series. Sepa-
rated multipath is consistent with the original series and
the denoised series in the low-frequency, which means that
TFM–CNN can effectively retain low-frequency multipath. Fig. 9  Power spectral density of each series in the east direction
Moreover, it has excellent robustness in the high-frequency,
but low-power noise remains in the frequency band higher
than 0.2 Hz. The bottom panel shows that the deformation deformation. We set up five sets of training data (DOY286,
separated by TFM–CNN no longer contains multipath in 292, 298, 304, 310) and analyze the remaining separation
the low-frequency. This method can accurately separate the deformation from the 20-day statistical results. We delineate
simulated deformation at the corresponding frequency. the correlation coefficient between separated multipath and
the original series and the RMS of their residual.
Experiment of different training data The boxplot shows the distribution of statistical results
and extreme changes. Figures 10 and 11 are shown the box-
To explore whether the parameter models obtained from plots of the correlation coefficient and RMS, respectively.
different training data affect the accuracy of separating For parameter models trained on different training data,
TFM–CNN has excellent performance for separating mul-
tipath. Moreover, the extreme value and the mean value of
the 20-day correlation coefficient and RMS are very simi-
lar. Table 1 shows the 20-day average correlation coefficient
and RMS of separating multipath from the five sets of the
parameter models. The correlation coefficients all have mini-
mal mutual differences on the order of ­10–3 and the mutual
differences of RMS are between 0.05 mm. Although the
amplitude and phase of different training data are different,
after the time-domain signals are converted into the spectro-
grams by STFT, 1D-CNN can learn the frequency-domain
characteristics of each frame. It may be the reason why the
TFM-CNN method still has the same precision separation
effect for different training data.

Experiment of training models in different

directions
Fig. 8  24-day correlation coefficient, RMS and bias between the orig-
inal series after mitigating multipath and the simulation deformation It is derived from the previous experiment whether the
in the east, north and vertical directions parameter model trained with only a specific direction

13
25 
Page 8 of 13 GPS Solutions (2021) 25:25

Fig. 10  Boxplots of the 20-day

correlation coefficients between
separation multipath and the
original series in the east, north
and vertical directions

Fig. 11  Boxplots of the 20-day

RMS after mitigating multipath
in the east, north and vertical
directions

Table 1  The 20-day average Train data East North Vertical

correlation coefficient and
RMS obtained from five sets of Correlation RMS (mm) Correlation RMS (mm) Correlation RMS (mm)
training data
286 0.944 0.425 0.957 0.427 0.958 1.162
292 0.938 0.447 0.953 0.444 0.958 1.163
298 0.943 0.432 0.957 0.424 0.958 1.164
304 0.940 0.439 0.958 0.423 0.958 1.204
310 0.942 0.434 0.958 0.422 0.957 1.176

series can be applied to separate multipath of the east, north
and vertical directions. For example, the parameter model
trained in the east direction can directly separate the east,
north and vertical directions. The three parameter models
separate multipath in the east, north and vertical directions
for the 24 days in the separation stage. Figure 12 shows the
evaluation index, which is the RMS of the residual between
the original series and the separated multipath.
We find that the three groups of models are consistent
in trend for the separation results in the east, north and
vertical directions (Fig. 12). However, there is a certain
difference between them, which shows that the robust-
ness of the three groups of models is different when
the multipath are separated. The E model retains some
high-frequency noise when separating multipath, while
the N model retains the least, indicating that multipath
separated by the N model are closer to the frequency-
domain characteristics of the true multipath. The statisti-
cal results in Table 2 show that the separation results of
the three models in the east, north and vertical directions
are similar and the maximum difference between the east
and north directions is only 0.011 mm and the maximum
difference between the three models in the vertical direc-
tion is 0.031 mm. It is only necessary to train the north
direction data to separate the multipath considering the
authenticity of multipath separation. That will not only
obtain excellent monitoring accuracy but also improve
network computing efficiency.

13
GPS Solutions (2021) 25:25 Page 9 of 13  25

were collected by five low-cost GNSS receivers (Model:

Fig. 12  24-day RMS of the separated multipath by three sets of the
parameter model
Experiment of different observation environments
Multipath is different in different observation environ-
ments. To verify the universality of certain training
data to separate multipath in different environments, the
experiment only uses the north direction training model of
DOY286 to separate multipath of the data under different
observation environments of DOY233 in 2020. The data
Hi-Target iRTK5) as rover stations in different environ-
ments, including cement surface, grassland, shrub, water
and building. The observation environments of rover sta-
tions are shown in Fig. 13. The base station uses the sta-
tion in the “Data” section. Receivers observed for 5 h with
the sampling rate of 1 Hz and the satellite cutoff angle is
15°.
Exact separation of multipath can improve the accuracy
of deformation monitoring. The universal applicability of
the TFM–CNN method in different environments is ana-
lyzed by separating multipath. To facilitate observing the
magnitude and separated multipath, the original series and
the separated multipath in the east direction are subtracted
from the average value of each series.
As shown the left panel in Fig. 14, the TFM–CNN method
mines the training data observed in different environments
to obtain the parameter model. The model still separates
multipath in the different observation series. When there
are float solutions with unfixed ambiguity in the observation
data (such as the data in the building observation environ-
ment), TFM–CNN can fit the multipath, which avoids the
influence of the float solutions on the separation multipath.
The residual between the five groups of observation data
in the right panel is stable and fluctuates at 0, which shows
that the influence of multipath is suppressed. However, the
deviation caused by the float solutions will be transmitted,
affecting the monitoring accuracy.

Table 2  The 24-day average Direction E model N model U model

RMS and bias of three sets of
the parameter model RMS (mm) Bias (mm) RMS (mm) Bias (mm) RMS (mm) Bias (mm)

East 0.430 1.128E−5 0.441 8.596E−6 0.435 5.891E−6

North 0.416 5.943E−6 0.427 − 9.165E−6 0.420 8.223E−6
Vertical 1.151 2.269E−4 1.182 1.958E−4 1.165 1.462E−4

Fig. 13  Observation environ-
ments and sites

13
25 
Page 10 of 13 GPS Solutions (2021) 25:25

Fig. 14  Original series and

separated multipath in different
environments (left) and residual
between the original series and
separated multipath (right)

The monitoring accuracy is different because multipath
and noise levels are different in different observation envi-
ronments. Table 3 shows the monitoring accuracy and the
degree of accuracy improvement before and after mitigat-
ing multipath. Excluding the building observation envi-
ronment, for the east and north directions, the original
monitoring accuracy is increased from millimeter level to
sub-millimeter level and the improvement of monitoring
accuracy is significantly improved; for the vertical direc-
tion, the improvement of monitoring accuracy is also more
apparent. However, multipath are serious in the building
observation environment. The monitoring accuracy is 6.575,
7.923 and 23.105 mm in the east, north and vertical direc-
tions. After mitigating multipath based on TFM–CNN, the
accuracy is significantly improved and the improvement is
70% ~ 79%. Moreover, affected by strong multipath, there are
a few centimeter-level float solutions in the series, limiting
the accuracy of deformation monitoring.
Conclusions
We proposed a joint time–frequency mask and convolu-
tional neural network method, suitable for real-time separa-
tion of multi-GNSS multipath. The method can be used to
obtain deformation in real-time. We analyzed the feasibility
of TFM–CNN to separate multi-GNSS deformation. Fur-
thermore, the method performs robust on different training
data and application environments as per our experiments.
Moreover, we analyzed the statistical results of the accu-
racy of multi-GNSS deformation monitoring after mitigating
multipath. Conclusions are as follows:

Table 3  RMS before and after mitigating multipath of the original series in different environments and their improvement. RMS means before
mitigating; RMS_M means after mitigating; Imp means the improvement
Env East North Vertical
RMS (mm) RMS_M (mm) Imp (%) RMS (mm)  RMS_M (mm) Imp (%) RMS (mm) RMS_M (mm) Imp (%)

Cement 1.260 0.836 33.68 1.931 0.907 53.06 3.870 2.236 42.22
Grass 1.489 0.867 41.76 1.516 0.838 44.73 4.024 2.450 39.11
Shrub 1.456 0.899 38.27 1.613 0.854 47.05 4.455 2.632 40.92
Water 1.614 0.955 40.86 1.904 0.940 50.61 5.029 2.769 44.93
Building 6.575 1.951 70.34 7.923 1.679 78.81 23.105 4.927 78.68

13
GPS Solutions (2021) 25:25 Page 11 of 13  25
1. TFM–CNN shows convincing performance in multi-
GNSS multipath separation, which also shows excellent
anti-noise robustness. For 24-day observation data, the
original monitoring accuracy is 1.3, 1.5 and 4.1 mm in
the east, north and vertical directions. The TFM–CNN
reduces the monitoring accuracy to 0.4, 0.4 and 1.2 mm
in our experiment. The low frequency of separated mul-
tipath is consistent with the original series. Moreover,
after mitigating multipath, the separated deformation is
highly consistent with the amplitude and frequency of
the simulated deformation even under the influence of
noise.
2. TFM–CNN seems not sensitive to the training data as
per our results. Moreover, only using training samples
in the north direction can accurately separate multipath
and improve the efficiency of network operations. That
is due to the network has learned the frequency-domain
characteristics of multipath in the training stage. It is not
limited by the frequency and amplitude of multipath.
3. We use the north direction of a baseline as training data
and obtain the parameter model for separating multipath
in the observation data collected by low-cost GNSS
receivers. Excepting the strong multipath environment,
the monitoring accuracy in the east and north directions
can be improved to millimeter level and the accuracy in
the vertical direction can reach 2.5 mm. For the strong
multipath environment, multipath can be significantly
mitigated and the monitoring accuracy can be improved
by more than 70%.
TFM–CNN method has excellent processing capabilities
for structural deformation monitoring in different environ-
ments and the method has broad application prospects. The
method is processed through waveform-frequency-waveform
and future work mainly needs to solve the signal separation
of waveform-waveform based on deep learning.
Acknowledgements  We would like to thank the two anonymous
reviewers for their constructive comments and suggestions. This
research is supported by the National Key Research and Development
Program of China (No. 2016YFC0803109), the Hebei Ecological
Intelligent Mine Joint Foundation (No. E2020402086), the National
Natural Science Foundation of China (No. 41772130) and the Anhui
University of Science and Technology Graduate Innovation Founda-
tion (2019CX2077).
Data Availability  Upon a reasonable request, the data that support the
findings of this study are available from the first author, Yuan Tao
(yuantao0505@gmail.com).
References
Agnew DC, Larson KM (2007) Finding the repeat times of the GPS
constellation. GPS Solut 11(1):71–76
Alber C, Ware R, Rocken C, Braun J (2000) Obtaining single path
phase delays from GPS double differences. Geophys Res Lett
27(17):2661–2664. https​://doi.org/10.1029/2000g​l0115​25
Brunner F, Hartinger H, Troyer L (1999) GPS signal diffraction mod-
elling: the stochastic SIGMA-Δ model. J Geod 73(5):259–267
Cai M, Chen W, Dong D, Song L, Wang M, Wang Z, Zhou F, Zheng
Z, Yu C (2016) Reduction of Kinematic Short Baseline Mul-
tipath Effects Based on Multipath Hemispherical Map. Sensors
16(10):1677
Choi K, Bilich A, Larson KM, Axelrad P (2004) Modified sidereal
filtering: Implications for high-rate GPS positioning. Geophys
Res Lett 31(22):178–198
Dai W, Huang D, Cai C (2014) Multipath mitigation via component
analysis methods for GPS dynamic deformation monitoring.
GPS Solut 18(3):417–428
Danskin S, Bettinger P, Jordan T (2009) Multipath mitigation under
forest canopies: A choke ring antenna solution. Forest Sci
55(2):109–116
Dong D, Wang M, Chen W, Zeng Z, Song L, Zhang Q, Cai M,
Cheng Y, Lv J (2016) Mitigation of multipath effect in GNSS
short baseline positioning by the multipath hemispherical
map. J Geod 90(3):255–262. https​: //doi.org/10.1007/s0019​
0-015-0870-9
Durak L, Arikan O (2003) Short-time Fourier transform: two funda-
mental properties and an optimal implementation. IEEE Trans
Signal Process 51(5):1231–1242
Genrich JF, Bock Y (1992) Rapid resolution of crustal motion at
short ranges with the Global Positioning System. J Geophys
Res Solid Earth 97(B3):3261–3269
Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hier-
archies for accurate object detection and semantic segmentation.
In: Proceedings of the IEEE conference on computer vision and
pattern recognition 2014, Columbus, Ohio, USA, pp 580-587
Groves PD, Jiang Z, Skelton B, Cross PA, Lau L, Adane Y, Kale I
(2010) Novel multipath mitigation methods using a dual-polar-
ization antenna. In: Proceedings of ION GNSS 2010, Portland,
OR, pp 140–151
Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdi-
nov RR (2012) Improving neural networks by preventing co-
adaptation of feature detectors. Preprint at https​: //arxiv​. org/
abs/1207.0580
Huang P-S, Kim M, Hasegawa-Johnson M, Smaragdis P (2015) Joint
optimization of masks and deep recurrent neural networks for
monaural source separation. IEEE/ACM Trans Audio Speech
Lang Process 23(12):2136–2147
LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-
based learning applied to document recognition. Proc IEEE
86(11):2278–2324
Luo Y, Mesgarani N (2019) Conv-tasnet: Surpassing ideal time–fre-
quency magnitude masking for speech separation. IEEE/ACM
Trans Audio Speech Lang Process 27(8):1256–1266
Maqsood M, Gao S, Brown TW, Unwin M, Steenwijk DVV, R,
Xu J, (2013) A compact multipath mitigating ground plane
for multiband GNSS antennas. IEEE Trans Antennas Propag
61(5):2775–2782
Nair V, Hinton GE (2010) Rectified linear units improve restricted
Boltzmann machines. In: Proceedings of the 27th International
Conference on Machine Learning (ICML), Haifa, Israel, pp
807-814
Phan H, Andreotti F, Cooray N, Chén OY, De Vos M (2018) Joint
classification and prediction CNN framework for automatic sleep
stage classification. IEEE Trans Biomed Eng 66(5):1285–1296
Pourtaghi A, Lotfollahi-Yaghin M (2012) Wavenet ability assess-
ment in comparison to ANN for predicting the maximum surface
settlement caused by tunneling. Tunn Undergr Space Technol
28:257–271
13
25 
Page 12 of 13 GPS Solutions (2021) 25:25

Ray J, Cannon M, Fenton P (2001) GPS code and carrier multipath Yuan Tao  is currently a graduate
mitigation using a multiantenna system. IEEE Trans Aerosp student in the School of Spatial
Electron Syst 37(1):183–195 Information and Geomatics
Srinivasan S, Roman N, Wang D (2006) Binary and ratio time-fre- Engineering at Anhui University
quency masks for robust speech recognition. Speech Commun of Science and Technology,
48(11):1486–1501 Huainan, China. He received his
Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R Bachelor’s degree from Anhui
(2014) Dropout: a simple way to prevent neural networks from University of Science and Tech-
overfitting. J Mach Learn Res 15(1):1929–1958 nology in 2014. His current
Wang D (2005) On ideal binary mask as the computational goal research interests include deep
of auditory scene analysis. Speech separation by humans and learning, multipath mitigation
machines. Springer, New York, pp 181–197 and high-precision GNSS data
Wang D, Chen J (2018) Supervised speech separation based on deep processing.
learning: An overview. IEEE/ACM Trans Audio Speech Lang
Process 26(10):1702–1726
Wang Y, Narayanan A, Wang D (2014) On training targets for super-
vised speech separation. IEEE/ACM Trans Audio Speech Lang Chao Liu  is an associate profes-
Process 22(12):1849–1858 sor in the School of Spatial
Wang Z, Chen W, Dong D, Wang M, Cai M, Yu C, Zheng Z, Liu Information and Geomatics
M (2019) Multipath mitigation based on trend surface analysis Engineering at Anhui University
applied to dual-antenna receiver with common clock. GPS Solut of Science and Technology,
23(4):104 Huainan, China. He received his
Weninger F, Eyben F, Schuller B (2014) Single-channel speech sepa- Ph. D degree in geodesy and sur-
ration with memory-enhanced recurrent neural networks. IEEE veying engineering from China
International Conference on Acoustics, Speech and Signal Pro- University of Mining and Tech-
cessing (ICASSP). Florence, Italy, pp 3709–3713 nology, Xuzhou, China, in 2011.
Xi R, Meng X, Jiang W, An X, He Q, Chen Q (2020) A Refined His current research interests
SNR Based Stochastic Model to Reduce Site-Dependent Effects. include GNSS signal processing,
Remote Sens 12(3):493 deformation monitoring and
Xu B, Wang N, Chen T, Li M (2015) Empirical evaluation of recti- mine survey.
fied activations in convolutional network. Preprint at https​://
arxiv​.org/abs/1505.00853​
Ye S, Chen D, Liu Y, Jiang P, Tang W, Xia P (2015) Carrier phase Chunyang Liu  is currently a lec-
multipath mitigation for BeiDou navigation satellite system. turer in the School of Spatial
GPS Solut 19(4):545–557. https ​ : //doi.org/10.1007/s1029​ Information and Geomatics
1-014-0409-1 Engineering at Anhui University
Yuan G, Xie Y, Song Y, Liang H (2010) Multipath parameters of Science and Technology,
estimation of weak GPS signal based on new colored noise Huainan, China. He received his
unscented Kalman filter. The 2010 IEEE International Con- Ph. D degree in geodesy and sur-
ference on Information and Automation. Harbin, China, pp veying engineering from China
1852–1856 University of Mining and Tech-
Zhang K, Li B, Zhu X, Chen H, Sun G (2017) Multipath detection nology, Xuzhou, China, in 2020.
based on single orthogonal dual linear polarized GNSS antenna. His current research interests
GPS Solut 21(3):1203–1211 include deep learning and spatio-
Zhang Q, Yang W, Zhang S, Liu X (2018) Characteristics of Bei- temporal big data analysis.
Dou Navigation Satellite System Multipath and Its Mitigation
Method Based on Kalman Filter and Rauch-Tung-Striebel
Smoother. Sensors 18(1):198. https​: //doi.org/10.3390/s1801​
0198
Zheng H, Fu J, Mei T, Luo J (2017) Learning multi-attention convo- Xingwang Zhao  is currently an
lutional neural network for fine-grained image recognition. In: associate professor in the School
Proceedings of the IEEE international conference on computer of Spatial Information and Geo-
vision 2017, Venice, Italy, pp 5209-5217 matics Engineering at Anhui
Zheng K, Zhang X, Li P, Li X, Ge M, Guo F, Sang J, Schuh H (2019) University of Science and Tech-
Multipath extraction and mitigation for high-rate multi-GNSS nology, Huainan, China. He
precise point positioning. J Geod 93(10):2037–2051 received the Ph.D. degree in the
Zhong P, Ding X, Yuan L, Xu Y, Kwok K, Chen Y (2010) Sidereal School of Instrument Science
filtering based on single differences for mitigating GPS multipath and Engineering at the Southeast
effects on short baselines. J Geod 84(2):145–158 University, Nanjing, China, in
Zimmermann F, Eling C, Kuhlmann H (2017) Empirical assessment of 2011. His current research inter-
obstruction adaptive elevation masks to mitigate site-dependent ests include GNSS precise posi-
effects. GPS Solut 21(4):1695–1706 tioning and network RTK.

Publisher’s Note Springer Nature remains neutral with regard to

jurisdictional claims in published maps and institutional affiliations.

13
GPS Solutions (2021) 25:25 Page 13 of 13  25
Haojie Hu  is currently a graduate
student in the School of Spatial
Information and Geomatics
Engineering at Anhui University
of Science and Technology,
Huainan, China. He received his
Bachelor’s degree from Anhui
University of Science and Tech-
nology in 2014. His current
research interests include GNSS
precise positioning and satellite
clock models.
Haiqiang Xin  received a Ph. D
degree in cartography and geo-
graphic information system from
Peking University, Beijing,
China, in 2014 and is currently a
GIS and Map Laboratory profes-
sor at Xinjiang Academy of Sur-
veying and Mapping, Urumqi,
China. His current research
interests include basic research
of surveying and mapping tech-
nology, remote sensing and geo-
graphic information system
technology.
13