Downloaded from https://iranpaper.

ir
https://www.tarjomano.com https://www.tarjomano.com

IEEE SENSORS JOURNAL, VOL. 22, NO. 23, 1 DECEMBER 2022 23095

Smartphone-Based Pedestrian NLOS

Positioning Based on Acoustics and
IMU Parameter Estimation
Hucheng Wang , Student Member, IEEE, Can Xue , Zhi Wang, Member, IEEE,
Lei Zhang , Student Member, IEEE, Xiaonan Luo , Senior Member, IEEE,
and Xinheng Wang , Senior Member, IEEE

Abstract —This paper proposes an integrated positioning

algorithm for mobile devices and achieves long-term and
high-accuracy indoor pedestrian tracking under severe non-
Line-of-Sight (NLOS) scenarios. The traditional fusion method
for hand-held devices lacks zero-speed correction and cannot
clear the accumulated error of the Pedestrian Dead Reckon-
ing (PDR). Secondly, the PDR algorithm also requires user
privacy data for high positioning accuracy. Hence, we pro-
pose a customized model with acoustic and PDR through
self-updating parameters with two novel fusing strategies:
Kalman Filter with Least-Square (KFLS) and Kalman Filter with
Bayesian Parameter Estimation (KFBPE), which utilize numer-
ical feedback and Bayesian distribution, respectively. Experi-
ments with Huawei Mate 9 show that both methods above can
effectively eliminate the outlier resulting from severe signal
loss, regardless of hand-holding gestures, with no individual
privacy data required. Extensive experimental results demon-
strate that the proposed methods are more efficient for NLOS
and perform much better than the baselines of traditional
fusion frameworks like the standard Kalman Filter. KFBPE
has a relatively smoother tracking result, which guarantees
an average positioning accuracy of up to 25 cm under the
circumstance of nearly thirty percent acoustic signal loss
(or NLOS) at the same time.

Index Terms — Inertial measurement unit, non-line-of-sight,

signal loss, step length.

Manuscript received 11 May 2022; accepted 17 June 2022. Date of I. I NTRODUCTION

publication 29 June 2022; date of current version 30 November 2022.
This work was supported in part by the Bagui Scholar Program Fund of
Guangxi Zhuang Autonomous Region of China under Grant 2019A40,
in part by the National Key Research and Development Program under
I NDOOR positioning plays a vital role in the fields of
robotics, smart homes, healthcare, and generally emerging
sector of the Internet of Things (IoT) [1], [2]. Over the years,
Grant 2018AA100305 and Grant 2021YFB3900800, and in part by the
National Natural Science Foundation of China under Grant 6202780103,
two broad categories of indoor positioning technologies have
Grant 61936002, and Grant 62033001. The associate editor coordi- been developed. One is infrastructure-based, where anchor
nating the review of this article and approving it for publication was nodes have to be deployed to provide positioning services.
Dr. Valerie Renaudin. (Corresponding author: Zhi Wang.)
Hucheng Wang is with the School of Computer Science and Informa-
The other is infrastructure-less such as positioning based
tion Security, Guilin University of Electronic Technology, Guilin 541004, on geomagnetic field [3] or inertial sensors. Results from a
China, and also with the College of Control Science and Engineering, series of Microsoft Indoor Localization Competition [4]–[6]
Zhejiang University, Hangzhou 310027, China.
Can Xue and Zhi Wang are with the State Key Laboratory of Indus-
events show that infrastructure-based positioning typically has
trial Control Technology, Zhejiang University, Hangzhou 310027, China better localization accuracy than infrastructure-free ones. How-
(e-mail: zjuwangzhi@zju.edu.cn). ever, these two categories have their spotlights and flaws.
Lei Zhang is with the School of Construction Machinery, Chang’an
University, Xi’an 710064, China.
Infrastructure-based positioning such as laser radar (Lidar)
Xiaonan Luo is with the Computer and Information Security School, and ultra-wideband (UWB) are usually accurate, reaching
Guilin University of Electronic Technology, Guilin 541004, China. a centimeter-level accuracy. However, the cost of hardware
Xinheng Wang is with the School of Advanced Technology, Xi’an
Jiaotong-Liverpool University, Suzhou 215000, China.
and deployment of anchor nodes is high. Infrastructure-free
Digital Object Identifier 10.1109/JSEN.2022.3185248 positioning does not need the deployment of anchor nodes,

1558-1748 © 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.

Authorized licensed use limited to: Universitaet Bielefeld. Downloaded on November 25,2023 at 11:32:49 UTC from IEEE Xplore. Restrictions apply.
Downloaded from https://iranpaper.ir
23096
which has lowered production costs. However, centimeter-level
accuracy can not be guaranteed. For location-critical applica-
tions, such as positioning for people with a sight impairment
or temporary isolation in COVID-19 [7], infrastructure-less
positioning is not suitable, despite its low cost to deploy and
easiness of operation.
Due to the importance of location-critical applications [8],
[9] and the pressure of high cost, one often takes the com-
bination of these two categories of positioning systems into
one integrated solution for granted. One possible solution is
to utilize the Inertial Measurement Unit (IMU) sensors. Com-
mercial off-the-shelf (COTS) intelligent terminal devices like
smartphones are becoming popular, so exploring smartphones’
indoor location is becoming potentially the best solution.
Since the breakthrough work of acoustic ranging with smart-
phones from Microsoft Corporation [10], acoustic localization
is experiencing a fast development, which can be found in
work around the world [4]–[6] and our lab [11]. For research
in recent years, the transmitted frequency band has shifted
from audible to inaudible. The application scenarios are moved
from Line-of-Sight (LOS) to NLOS scenarios, the positioning
target is moving dynamically, and the signal is moving dynam-
ically transmission has been changed from active broadcast
to passive reacting. These developments have made acoustic
positioning with high performance in complex environments.
These advances can also be found in a series of indoor local-
ization competitions. Our lab’s results in the 2018 Microsoft
Indoor Localization Competition showed acoustic localization
could reach 70 cm in practical applications [5], which put us
at the first two places in the sound group. If augmented real-
ity (AR) is implemented to support the localization, accuracy
of acoustic localization can reach 50 cm [12]. The positioning
errors do not exceed 30 cm in the LOS scenario by deep
analyzing the positioning data.
In contrast, it rises to meter-level in the NLOS scenarios.
To reduce the NLOS error, data fusion from the IMU and
acoustic localization was employed. The localization error
could be maintained within 28 cm by applying the Extended
Kalman Filter (EKF) to eliminate the acoustic outliers under
LOS and NLOS scenarios in [13]. However, under location-
critical scenarios, they lack systematic analysis of acoustic
outliers. An in-depth analysis and better solution are needed to
ensure that the positioning system performs well under various
scenarios.
In this paper, we have investigated near-ultrasonic Time
Difference of Arrival (TDoA)-based localization and IMU nav-
igation for positioning under severe NLOS scenarios because
(i) near-ultrasonic is inaudible, which causes no interference
to human beings, (ii) NLOS scenario is more common than
LOS scenario, which almost exists everywhere, (iii) TDoA
eliminates dependence on precise synchronization that nei-
ther Android nor iOS phones support, (iv) the accuracy of
short-term Pedestrian Dead Reckoning (PDR) is relatively
reliable, and (v) it will be widely used rapidly because both
technologies can be implemented in smartphones.
In practice, relying on near-ultrasonic signals alone for
positioning is unreliable because of the NLOS phenomenon
and its properties of signals being easily interfered with,
Authorized licensed use limited to: Universitaet Bielefeld. Downloaded on November 25,2023 at 11:32:49 UTC from IEEE Xplore. Restrictions apply.
https://www.tarjomano.com https://www.tarjomano.com
IEEE SENSORS JOURNAL, VOL. 22, NO. 23, 1 DECEMBER 2022
absorbed, and the positioning deviation. Furthermore, to obtain
the three-dimensional indoor positioning accurately, the TDoA
algorithm requires at least four beacon-to-phone channels to be
unobstructed. However, the acoustic signal is often blocked in
complex indoor environments, making the TDoAs suffer great
disturbance.
This paper focuses on mitigating the deviation of acoustic
localization by data fusion of acoustic localization and PDR
with its short-term high-precision characteristics by applying
a tight coupling approach at each step of the fusion procedure.
It is well-known that there are two kinds of PDR mod-
els [14], the general model and the customized model, respec-
tively. The former model’s performance is relatively poor,
while the latter has to use personal data, which might cause
privacy issues. In [15], several customized models for step
length (S L) were mentioned as Eq. (1), where l is leg length,
T is step duration, h involves height and step frequency, the
parameter K is derived from a default value to calibrate the
estimator for users. The first equation only takes into account
body proportions. In contrast, the second equation considers
the vertical accelerations avert during the step, and the range
a peak,di f f of the vertical accelerations. The third equation is
another empiric method that is based on the correlation of
the vertical acceleration at the foot a f oot,vert during one step
with the length of the step. When it comes to location-based
apps, collecting private data from users is bad. During the
fusion process, the acoustic localization is applied to reduce
the accumulated error of PDR and estimate the PDR general
model’s individual feature coefficient.

S L = K 2lh − h 2
 
 N
 |a | K
S L = 0.1 
vert,i
2.7 i=1

N T · a peak,di f f

N
3
i=1 |a f oot,vert,i |
S L = 0.98 (1)
N
Meanwhile, by utilizing the PDR’s property of short-term
stability, the temporary jitter of acoustic localization is elimi-
nated. Combining acoustic and PDR’s superiority can improve
overall positioning performance while all-time high-precision
positioning is achieved.
Our approach: We have introduced two methods in this
paper to increase positioning precision, namely (i) the numer-
ical coefficient in each step in Least-Square after Kalman Filter
fusion (KFLS), and (ii) probability distribution coefficient in
each step for the Bayesian Parameter Estimation (KFBPE).
Then we consider outliers with a certain probability. Bayesian
Parameter Estimation can smooth outliers with prior knowl-
edge, so KFBPE is proposed. Experiments show that under the
condition of the acoustic position loss rate being higher than
30%, the accuracy of up to 25 cm in mean square error (MSE)
is achieved. In summary, this paper has the following technical
contributions to knowledge, including:
• a complete real-time fusion positioning system that com-
bines the advantages of acoustic positioning and PDR,
which is well-integrated into COTS smart devices,
Downloaded from https://iranpaper.ir
WANG et al.: SMARTPHONE-BASED PEDESTRIAN NLOS POSITIONING BASED ON ACOUSTICS AND IMU PARAMETER ESTIMATION
• KFLS and KFBPE methods are proposed to improve per-
sonal coefficients based on online self-learning, without
privacy issues, and
• even if an external position loss rate is up to 30%, we can
still derive accurate positioning.
The remaining parts of this paper are organized as follows:
Related work is described in Section II. Then we present our
approach and provide the details for its implementation in
Sections III and IV, respectively. Performance evaluation is
conducted in Section V. Finally, Section VI concludes this
paper.
II. R ELATED W ORK
Numerous Indoor Positioning Systems (IPS) have been
proposed to meet the increasing demand for Location-Based
Services (LBS). Traditional indoor positioning technologies
can be generally classified into several categories: Radio
Frequency (RF), optical, acoustic, and mechanical. Table. I
presents a summary of different IPS, comprehensively depict-
ing the accuracy, cost, coverage, and compatibility of portable-
device. Unlike the Global Navigation Satellite System (GNSS)
dominant technology used in open space, no such dominant
technology is used indoors. Each indoor localization technol-
ogy has its inherent advantages and disadvantages. We will
briefly describe their advantages and drawbacks in the rest of
this section.
RF-based indoor positioning methods were once treated as
the most promising IPSs for their strong compatibility and
relatively low cost. Among them, Wi-Fi undoubtedly claims
the most attention, with many problems to overcome, in any
case. Chen [16] developed a large number of fingerprint
databases with Channel State Information (CSI) detection,
realizing the commercialization of indoor positioning in shop-
ping malls and airports. Nevertheless, the accuracy of Chan’s
method still failed to break through the meter-level. Hence,
it can hardly meet the request of a high-accuracy situation.
Researchers in [17] use WiDraw to implement an in-air
handwriting application that allows the user to draw letters,
words, and sentences and achieves a mean word recognition
accuracy of 91%. Researchers are also working to reduce the
number of base stations. PILSBON used iterative numerical
estimation to compensate for the nonlinear errors and deployed
an effective positioning system with six transmitters on the
rooftop [18]. To modulate the 2.4 GHz carrier frequency,
PILSBON requires a 46.43 MHz intermediate frequency gen-
erator, raising the total cost. Other RF-based methods like
Bluetooth Low Energy (BLE) suffer from poor accuracy, high
installation cost, or both.
In comparison, UWB appears to be the more accurate one.
The positioning accuracy of UWB can reach a centimeter-level
with a frequency of approximately 3.1-10.6 GHz. Although
it has been demonstrated that UWB can penetrate walls in
buildings and resolve multi-path effects due to its wide band-
width, it is still a challenge to employ it in indoor environ-
ments. UWB also seems to be advantageous in scalability.
Unscented Kalman Filter (UKF) was used in [19] to combine
UWB landmarks and inertial sensors and enhance the tracking
accuracy. In the same way, AirTag was developed at Apple Inc.
Authorized licensed use limited to: Universitaet Bielefeld. Downloaded on November 25,2023 at 11:32:49 UTC from IEEE Xplore. Restrictions apply.
https://www.tarjomano.com https://www.tarjomano.com
23097
to replace the long-existing iBeacon [20], known for its low
power consumption. Some iOS devices support UWB and are
said to be compatible with AirTags. The news may have a
specific impact on the IPS industry.
Unlike RF-based positioning methods, optical-based IPSs
are widely used in the industrial field. Representative tech-
niques have been successfully developed and popularly
applied, such as LIDAR and Visible Light Communications
(VLC). Wolfgang [21] presented the LIDAR in a Simultaneous
Localization and Mapping (SLAM) platform, which achieved
real-time mapping and loop closure at a 5 cm resolution.
In practice, LIDAR is more expensive than UWB and is not
supported on mobile phones, so it is unlikely to be widely used
by the public. Visible light is also a medium commonly used
for positioning. An image sensor-based indoor visible light
positioning system by Singular Value Decomposition (SVD)
and enhanced robustness was proposed in [22]. Tango [23]
leveraged the ubiquitous, always-on Augmented Reality (AR)
glasses and collected camera data from the lenses. The
project was soon abandoned for privacy-preserving and power-
consumption concerns, which this paper will carefully deal
with.
Mainly used in the COTS field, an acoustic-based position-
ing system (APS) has already become a research hotspot. APS
is generally known for its relatively low cost and broad cover-
age with the most reliable compatibility in intelligent devices.
Beepbeep [10] emitted the double-beep sounds during a rang-
ing session by Round-Trip Time (RTT). Beepbeep artfully
avoided many sources of inaccuracy found in other typical ToA
schemes. BatTracker [25] was proposed to achieve the high
precision and infrastructure-free mobile device tracking system
in 3D space. BatTracker accommodates uncertainty based on
a probabilistic algorithm for creating, pruning, and evolv-
ing multiple hypotheses. Speaker-to-Microphone (S2M) [30]
was an acoustic fingerprint method to authenticate devices.
In [31], SteerTrack for steering tracking can track the rota-
tion angle in real-time, leveraging audio devices on smart-
phones. For acoustic signal selection and processing, the lin-
ear chirp signal is employed in a dense multi-path environ-
ment with Fractional Fourier Transform (FrFT) [11]. RAM-
TEL [26] was able to achieve a millimeter-level accuracy
for localization and motion tracking in multi-path fading by
leveraging phase measurement. However, the efficiency is
significantly reduced in the occlusion conditions. A novel
localization system that fused the position information esti-
mated by acoustic localization and IMUs was proposed in [32].
Researchers in [24] improved the accuracy of ultrasonic direc-
tion angles and applied it to SLAM. ALPS [12] was very
considerate in NLOS recognition, Doppler Effect, and floor
plan while ignoring the motion tracking problem without
IMU utilization. Considering the most significant difference
between ALPS and our paper, the system proposed by us can
dynamically position the moving target with a novel fusion
strategy.
The IMU-based system is a classical method for
infrastructure-less localization. In general, the pre-condition
of the high-precision IMU strategy is to make the sensors
perfectly fit the mover during movement. Without such a
Downloaded from https://iranpaper.ir
https://www.tarjomano.com https://www.tarjomano.com
23098 IEEE SENSORS JOURNAL, VOL. 22, NO. 23, 1 DECEMBER 2022

TABLE I
C OMPARISON OF I NDOOR P OSITIONING M ETHODS

pre-condition, errors may be reduced to at least 1-2 meters

in the Indoor Positioning and Indoor Navigation (IPIN) com-
petition dataset [33]. Exacerbating the problem, the IMU
in our paper differs from most others in that we mainly
apply embedded IMUs of a smartphone. Considering the
smartphone’s cost, the COTS IMU was lightweight, and its
tracking error accumulated rapidly over time with double
integration. Zero Velocity Update (ZUPT) corrects speed with
standstill determination to avoid the error growth by double
integration. When the ZUPT detects that the user is stationary,
the velocity calculated by the double-integration seems to
be the system error’s observation. [27]. ZUPT requires that
specific sensors be mounted on foot, then divides pedestrian
walking into several processes. When the left and right feet are
switched, the foot which supports the body is static relative to
the ground (meaning zero velocity), so the cumulative error
can be corrected in each step. In practice, phones are often
placed in hands, pockets, or bags rather than mounted on the
waist or foot, which is irrational for commercial use. Authors
in [28] realize pedestrian tracking for unconstrained carrying
the smartphone in all gestures by identifying three typical
modes in real-time. Since [28] has accomplished a set of
complete work in multi-mode PDR, this paper will mainly
focus on the situation with handhold gestures. The PDR in
IMU has a visible effect on plane positioning, but there is still
a significant error in the vertical direction. A barometer was
used to deal with vertical positioning errors with the Bayesian
Network inference method [29].
To summarize the methods mentioned above, none of them
can give the ground and stable location alone. Combin-
ing sounds and IMU has been discussed in a few papers.
Researchers in [32] ingeniously proposed the Kalman filter
method. It integrated the advantages of both approaches and
obtained fusion signals with tight coupling, which led to better
stability and accuracy. However, it can only temporarily deal
with a single NLOS scenario but not solve the problem of
regional signal loss (or NLOS) and improve PDR estimation
performance.
For standard Kalman filters, the consideration for acoustic
signal loss is insufficient. We have improved the algorithm
in this paper, taking the fused locations as the input value
and solving the modified step coefficient according to the
least-squares method. CAT in [34] combined the Doppler
Shift, the IMU, and acoustic characteristics to achieve the
millimeter-level near-field positioning (coverage within one
meter). Authors in [26] further refined the CAT accuracy
by employing phase ranging using the Extreme Learning
Fig. 1. Diagram of proposed methodology.
Machine (ELM) method and detected breathing successfully.
However, they consume too much battery energy emitting and
receiving the uninterrupted Frequency Modulated Continuous
Wave (FMCW) and could not work in far-field positioning
even under LOS conditions.
III. S YSTEM A RCHITECTURE
This paper has proposed a novel approach that is fully
automated by using existing COTS devices. The schematic
diagram of the method is shown in Fig. 1. The most significant
advantage of this approach is that it can be directly applied to
smartphones without extra hardware, except signal emission
beacons need to be placed by the location service provider in
advance.
A. TDoA Based Acoustic Localization
We have implemented our system in a real-world environ-
ment. The acoustic TDoA passive positioning is adopted, and
the conceptual architecture is shown in Fig. 2. The synchro-
nization of inter-beacon is modulated with 433 MHz. The
beacons are fixed, and the phone is held in hand. Different
beacons transmit chirp signals in a Time Division Multiple
Access (TDMA) mode. The proposed algorithm is imple-
mented on the phone. Then the phone collects acoustic signals
and IMU data without using other devices.
The beacons are usually fixed on walls or ceilings to emit
modulated acoustic signals. The microphone embedded in the
smartphone receives and demodulates signals with a General-
ized Cross-Correlation PHAse Transformation (GCC-PHAT)
algorithm [35], and calculates the Time of Flight (ToF). Based
on the calculated ToF, the distance between the beacons and
the phone can be obtained, and the indoor locations of the
phone are determined.

Authorized licensed use limited to: Universitaet Bielefeld. Downloaded on November 25,2023 at 11:32:49 UTC from IEEE Xplore. Restrictions apply.
Downloaded from https://iranpaper.ir
WANG et al.: SMARTPHONE-BASED PEDESTRIAN NLOS POSITIONING BASED ON ACOUSTICS AND IMU PARAMETER ESTIMATION
Fig. 2. Conceptual architecture of acoustic TDoA passive positioning.
Fig. 3. (a) The performance of our acoustic prototype in 2018 Microsoft
Indoor Localization Competition. (b) The outlier data are circled in black.
The error analysis of competition results.
TABLE II
T HE A NALYSIS OF C OMPETITION R ESULTS
B ASED ON LOS AND NLOS C ONDITIONS
aco is calculated as
The acoustics positioning Xm
ctm,i1 = Xm
aco
− Bi 2 − Xm
aco
− B1 2 +ψi , i ≥ 2, (2)
 We analyze the Allan variance curve of the tri-axial average
T Do Am,i1
where ψi ∼ N (0, σi2 ),  • 2 is Euclidean distance in 2-norm,
and subscript i 1 means from Beacon Bi to Beacon B1 in
TDoA. Assume that the speed of sound c is constant at
340 m/s. tm,i1 is derived from the GCC-PHAT of received
acoustics in m step and i beacon, and not completely accurate
due to the random loss and NLOS in acoustic signal propaga-
tion. Bi means the position of i th beacon and B1 denotes the
position of 1st beacon. The first beacon is set as the principle
beacon, and subtracted by other beacon. Eq. (2) can be solved
by Chan in [36].
B. Performance Analysis of Near-Ultrasonic Positioning
A typical acoustic positioning system is composed of a
smartphone and more than four loudspeakers. The loudspeaker
emits modulated acoustic signals as a beacon, and the smart-
phone receives modulated signals as a tag and calculates the
TDoA in real-time.
Figs. 3(a), 3(b) and Table. II show the results and analysis
of the 2018 Microsoft Indoor Localization Competition [5],
where the blue curve is the trace of ground truth, the green
Authorized licensed use limited to: Universitaet Bielefeld. Downloaded on November 25,2023 at 11:32:49 UTC from IEEE Xplore. Restrictions apply.
https://www.tarjomano.com https://www.tarjomano.com
23099
spots represent the estimated positions of our developed
acoustic positioning, the red line indicates the errors. The
black circle labels the severity of outliers. The competition
arena is a double-floor building, including a hall, corridor, and
stairs. The Lidar scheme gives the ground truth. We deployed
ten beacons distributed evenly in the test area. As can be
seen from Fig. 3(a), except for a few outliers, which are
extremely far away, most of the other errors are tolerable.
Due to the inaccurate sound velocity calibration, the outer-ring
positioning results are generally smaller than the ground truth.
Then, the signal loss and errors from the competition results
are analyzed in Table. II. It can be seen that most of the
positioning results are relatively accurate except for one severe
error drifting from the outlier point. Floor shielding produces
the NLOS area, resulting in the loss of acoustic signals and
outliers.
C. IMU Offset and Orientation
We employ the nine-axis IMUs embedded in Android
smartphones for the inertial sensor, including gyroscopes,
accelerometers, and magnetometers. The magnetometer deter-
mines the initial direction, the accelerometer and gyroscope
jointly measure the angular velocity, and the accelerometer is
mainly used to measure acceleration.
We collect the acceleration and angular velocity data from
the smartphone with a sampling frequency of 30 Hz. Because
these data are calculated with a double integration, even the
minor systematic deviation will be magnified. To calibrate the
minor offset of the phone IMU system, we have analyzed
the sensor data with the help of Allan variance modeling [37].
The Allan variance modeling measures the stability of pre-
cision instruments and identifies various noise sources present
in fixed sensor measurements. Data of length of four hours is
collected with a sampling frequency of 100 Hz, and the results
are shown in Fig. 4.
in segments from the log-log curve shown in Fig. 4. The
random angle walk is defined as the white noise spectrum of
the IMU output, shown in a black line with a slope of z0.5 in
Fig. 4. The random rate walk is defined as the Brownian
noise of the IMU output, shown in a black line with a slope
of 0.5. Since the vertical axis has been adjusted, the slope
does not match the actual log-log curve. Moreover, the line of
slope : 0 represents the data of bias instability, which is the
source of the Flicker noise of IMU.
Then we amend the sensor bias as
â(t) = a(t) + bi as Na (t) + bi as aB (t) + bi as Ka (t)
ω̂(t) = ω(t) + bi as Nω (t) + bi as ωB (t) + bi as Kω (t) (3)
where â(t) and ω̂(t) denote the raw accelerometers and gyro-
scopes reading with bias respectively. The subscripts N, B, K
represent angle random walk, bias instability, and rate random
walk.
The raw gyroscopes ω after bias removal are also not
handled directly and require conversion. Euler Angle is the tra-
ditional way of estimating the rotation angle except for Gimbal
Lock. To avoid Gimbal Lock, the quaternion is employed to
Downloaded from https://iranpaper.ir
https://www.tarjomano.com https://www.tarjomano.com
23100 IEEE SENSORS JOURNAL, VOL. 22, NO. 23, 1 DECEMBER 2022

Fig. 5. The comparison with right hand and left foot, while handhold
data are slight and delay.

where ait = [0, ât,x , ât,y , ât,z ]T is denoted tri-axis accelerome-

ter reading by reduced bias. The value of ait is always affected
by gravitation constant, so gg = [0, 0, 0, g] is subtracted
in Eq. (9).

D. Step Judgment
After calibrating the acceleration and rotation, we have
two options: acceleration to displacement, separately double
integration method with Inertial Navigation System (INS),
Fig. 4. Acceleration and angular velocity in Allan deviation of the log-log
curve. Tri-axial and average data are shown.
and PDR method. Since the acceleration data are inaccurate,
double integration is equivalent to double-expansion offset.
update orientation. A convert quaternion can be regarded as a Then we select the PDR method to obtain displacement to
hyper-vector as q = [q0 , q1 , q2 , q3 ]T , usually the initial value avoid double integration. The test phone is hand-held instead
of q0 is zero. The first-order differential equation gives the of tied to our system’s human body or feet. Whether walking
generation of q as or calling, the sensor data always contains buffer and delay
from the wrist and shoulder, so the sensors reading can hardly
dq
= i q, (4) reflect the actual posture. Fig. 5 compares hand-held phone
dt ⎛ ⎞ data in red and foot-tie-phone data in purple. The handhold
0 −ω̂x −ω̂ y −ω̂z data has a significant delay compared to the foot data. In line
1⎜
⎜ω̂x 0 ω̂z −ω̂ y ⎟
⎟
 = ⎝
i
(5) with the kinematics model, it satisfies the following conditions
2 ω̂ y −ω̂z 0 ω̂x ⎠ after removing delay.
ω̂z ω̂ y −ω̂x 0,
|a f oot | ≥ |ahand | (10)
where ω̂ = [0, ω̂x , ω̂ y , ω̂z ]T is gyroscope amendment reading.
Then q is updated by first order Runge-Kutta Principle as where a f oot is the data of IMU on foot, whereas ahand is the
data of IMU on hand.
qt +1 = exp(it dt)qt . (6)
At first, we should distinguish each step boundary. In Fig. 6,
Ri and R g are denoted IMU system and geodetic frame the orange line shows the raw accelerometer data, and the
system transformed by thick red line shows data after the average smooth filter
  while the window size is set to 5. Then the peak threshold
Rg = q Ri qT = Q(Ri ). (7) is set in advance in the gray dotted line. Once the red line
We express transform as Q(•). The Eq. (8) is none-matrix peak exceeds, the threshold is considered a practical step.
multiplication, but Hamilton products. Furthermore, the peak-peak interval cannot be too dense [38].
 Otherwise, it will still be justified as a jitter.
[w1 , x 1 , y1 , z 1 ] [w2 , x 2 , y2 , z 2 ] As far as we know that step length is individual from
= [w1 w2 − x 1 x 2 − y1 y2 − z 1 z 2 , person to person, and the experiments should be conducted
w1 x 2 + x 1 w2 + y1 z 2 − z 1 yz 2 , in different populations. However, due to the arbitrariness and
complexity of human movement, the threshold and segment
w1 y2 − x 1 z 2 + y1 w2 + z 1 x 2 , may still not accurately judge the pace. For example, when
w1 z 2 + x 1 y2 − y1 x 2 + z 1 w2 ] (8) the arm lifts the smartphone straight, it is almost uniformly
moved, resulting in nearly zero acceleration. The loss is
The accelerator reading from i to g is converted as
inevitable unless the phone is tied directly to the shoe (like
g
at = Q t (ait ) − gg , (9) ZUPT). Merely changing the threshold for acceleration may

Authorized licensed use limited to: Universitaet Bielefeld. Downloaded on November 25,2023 at 11:32:49 UTC from IEEE Xplore. Restrictions apply.
Downloaded from https://iranpaper.ir
https://www.tarjomano.com https://www.tarjomano.com
WANG et al.: SMARTPHONE-BASED PEDESTRIAN NLOS POSITIONING BASED ON ACOUSTICS AND IMU PARAMETER ESTIMATION 23101

Fig. 7. The event points of acoustics and PDR. The frequency of PDR
may be non-constant. The point of acoustics will lost occasionally.

Fig. 6. Example of step judgment. The peak marked with a red
pentagram represents a step of the pedestrian. The black dotted line
denotes the threshold, and the green dotted circle below the threshold
is a missing step. The blue line shows the v H of the missing step.
A. Strategy of Fusion
We assume the ground truth state at time k is evolved from
the state k − 1 by PDR model from Eq. (13)

Xk+1 = Xk + Sk T · uk + wk , (13)
be counterproductive. For example, we set the cut-off peak
at 0.5 m/s 2 , while the lost peak in Fig. 6 appears in around
0.3 m/s 2 , which even lower than some sub-peaks. A false where Xk = [x k yk ]T is denoted as plane coordinate at k in
peak method was proposed in [39], except for dealing with indoor frame system, Sk is denoted as step length in k, and
the lost peak. We found that the velocity within a single step uk = [sin θ cos θ ]T is controller vector where θ is azimuth
is calculated as Eq. (11). angle from Eq. (7). The PDR noise which follows a normal
distribution with wk ∼ N (0, Qk ).
vtH = vtH−1 + atH t, (11) The indoor frame system usually obeys the right-hand rule,
and the building determines the orientation. The orientation
where H is denoted as a horizontal component, for instance leads to a transformation between the indoor frame system
y
atH = [(atx )2 , (at )2 ]. t is the IMU sampling interval, shown and the geodetic frame system, denoted as T in
in Fig. 6. The threshold in Eq. (11) is assigned by practical
experience.  
cos φ − sin φ
T= , (14)
sin φ cos φ
E. Step Length Measurement
The traditional PDR method implements personal data, where φ is the rotation angle.
including but not limited to height, age, and leg length, to gen- The observation matrix in our method is different from
erate crucial parameters, raising tremendous privacy concerns. the usual Kalman filter. We define the moment of received
Authors in [40] raise constant paces for different motion acoustics as tm and of walk step tk . Unfortunately, the fre-
modes, including walking, jogging, and running. We propose a quency of pace is not equal to acoustic frequency, as shown
feedback step length estimation. Initially, an empirical formula in Fig. 7. We set a time gap interval (tgap ), when |tm − tk | is
as Eq. (12) designed by Weinberg describes the corresponding less than tgap , tm is approximately the same as tk . Otherwise,
relationship between step length and peak-valley of accel- when |tm − tk+1 | is less than tgap , this moment is considered
eration [41]. Weinberg only needs a fourth-root calculation to be none of observation, so skip to k + 1. If else, tm is
compared with other models, and it can be accelerated by just between tk and tk+1 , and we estimate the data of Xkaco by
Newton’s Method with rapid iteration. first-order Newton interpolation. The synchronization criterion
of acoustic signal and PDR is shown as Eqs. (15) and (16).
g g 1
Sk = γ (|ak |max − |ak |min ) 4 (12) ⎧
g
⎪
⎨Xm , |tm − tk | ≤ tgap ,
aco
|ak |is denoted as the estimation scalar of acceleration in k
Xkaco = skip, |tm − tk+1 | ≤ tgap , (15)
step, γ represents the correction parameters of an individual. ⎪
⎩
In fact, the preset value γ is not suitable for everyone in αk Xm
aco + (1 − α )Xaco , other wi se,
k m−1
the motion model. γ cannot be collected, which involves the tk − tm−1
αk = . (16)
privacy of personal body type [42]. It is also impractical to tm − tm−1
modify it by handling IMU value again. In order to protect
user privacy, we did not collect individual data to obtain the Then, we define the observation matrix as
step coefficient. Considering many loss data in observation,
we propose employing the Kalman Filter to compose IMU zk+1 = Xk+1
aco
= Xk+1|k + vk , (17)
and acoustic data.

where vk ∼ N (0, Rk ). Due to the uncertain error distribution

IV. S TEP L ENGTH C ORRECTION B ASED F USION of the acoustics, the outliers caused by signal loss or NLOS
In Weinberg’s model, the step coefficient γ is determined by are considered unsuitable for fusion. Therefore, we need to
prior knowledge such as the user’s height, leg length, gender, predict the acoustics positioning certainty L as to the weight.
and weight; all the data are users’ privacy. The advantage of We calculate the L and  L from Xm aco of Eq. (2) to Eqs. (18)

our approach is self-iteration without privacy problems. and (19) into the updating process of weight by a Kalman

Authorized licensed use limited to: Universitaet Bielefeld. Downloaded on November 25,2023 at 11:32:49 UTC from IEEE Xplore. Restrictions apply.
Downloaded from https://iranpaper.ir
23102
filter, where the σi is from ψi ∼ N (0, σi2 ) in Eq. (2).
aco
L(Xm |Bi , · · ·, Bn )
n
1 (ctm,i1 − T Do Am,i1 )2
= √ ex p{− }, i ≥ 2,
i=2
2πσi 2σi2


L
, if L < L max ,
L = L+1
1, if L ≥ L max .
In general, L max is the maximum likelihood, which is set
aco is considered loss and
to 100. When L exceeds L max , Xm
will not be updated z.
Then Kalman gain Kk+1 , Xk+1 at state k + 1, and covari-
ance Pk+1 from (20–23) are updated. The measurement H is
temporarily considered as a 2-order identity matrix.
Kk+1 = Pk+1|k H T (HPk+1|k H T + Rk+1 )−1 , (20)
Xk+1 = Xk+1|k + Kk+1 [(1 −  L k+1 )yk+1 + L k+1 Xk+1|k ],
yk+1 = zk+1|k − HXk+1|k ,
Pk+1 = (I − Kk+1 H)Pk+1|k .
So far, we have estimated the actual movement path of
the target, but it is still based on a free acoustic propagation
channel. It is practically impossible to maintain such an ideal
situation all the time. In other words, the acoustic signal is
often lost, or the observation data sometimes are incredibly
inaccurate. We can only trust IMU at that moment. However,
neither the issue of step coefficient nor accumulated error
in IMU is resolved. The next subsection task guarantees
the personal step coefficient through existing fusion data,
videlicet, guarantees long-term accuracy without relying on
stable acoustic data, and eliminates the accumulated error.
B. Updating γ From KFLS
As mentioned in Eq. (12), Weinberg’s pace estimation
involves a specific parameter of γ . In order to update γ ,
we push back step length with Least Square after Kalman
Filter according to the fusion coordinates. Assuming a con-
secutive two-step model based on Eq. (13), the cost function
of Sk is denoted as
Js (Sk ) = Xk+1 − Xk − Sk T · uk 22 .
To minimize the cost function Ŝk = arg min Js (Sk ), the step
Sk ∈R
length estimation Ŝk must satisfy
Ŝk = (T · uk )† (Xk+1 − Xk ),
where † denotes the pseudo-inverse matrix. Then γ is calcu-
lated as (26) on the basis of (12)
1
γ̂ = Ŝk /(|âk |max − |âk |min ) 4
KFLS accomplishes the Least Squares (LS) specific parame-
ters update γ through the above process. Generally speaking, γ
is relatively stable for the same person with the same posture,
with maybe no change in the short term.
Authorized licensed use limited to: Universitaet Bielefeld. Downloaded on November 25,2023 at 11:32:49 UTC from IEEE Xplore. Restrictions apply.
https://www.tarjomano.com https://www.tarjomano.com
IEEE SENSORS JOURNAL, VOL. 22, NO. 23, 1 DECEMBER 2022
C. Updating γ From KFBPE
KFLS method is calculated and overwritten when a new
fusion position is obtained. Once the user pace is unstable, the
step coefficient will endure a substantial deviation from the last
datum, KFLS also changes dramatically. Because the method
(18) only considers single loss signals, they will be eliminated by
the next fusion even if upheaval appears. In KFBPE, we are
caring about large-scale loss signal problems. A Bayesian
(19)
estimation-based step coefficient correction scheme is adopted,
which significantly enhances the robustness of positioning.
We consider that positioning is a Hidden Markov
Model (HMM) model, that the transition of each location
depends on the last n locations. So γ probability model is
simplified as P(γ |Xk , Xk−1 , . . . , Xk−n+1 ) = P(γ |X K ), (k ≥
n), according to Bayesian estimation as
P(X K |γ )P(γ )
P(γ |X K ) = ∝ P(X K |γ )P(γ ) (27)
P(X K )
(21) where from Xk to Xk−4 are positions we obtained in Kalman
Filter, prior probability P(γ ) is known by variable PDR
(22) database. Then we calculate the likelihood P(X K |γ ) as
(23)
P( K |γ )P(X K | K,γ)
P(X K |γ ) = (28)
P( K |X K , γ )
where K is a set of observation data Xaco K , likelihood
of acoustics L K , and IMU data, which is independent of
γ . Then we simplify Eq. (28) and eliminate the evidence
P( K |X K , γ ),
P(X K |γ ) ∝ P( K )P(X K |γ , K ). (29)
Now we have a prior probability of P( K ) from IMU and
acoustics, and just calculate P(X K |γ , K ) from Kalman Filter
in Eqs. (21) and (22) as Eq. (30), as shown at the bottom of
the next page.
It means the state value Xk+1 update is regarded as
the weight of observation and prediction, and correspond-
B P E and X B P E . Calculate the
ing weights are respectively Zk+1 k+1
spectral radius and normalize them of ρ(Zk+1 B P E ), ρ(X B P E ) to
k+1
approximate the probability P(z K |γ , K ) and P(X K |γ , K ).
Finally, we integrate Eqs. (27) and (29),
P(γ |X K ) ∝ P( K )P(X K |γ , K )P(γ )
(24) = P( −1
K )P(X K |γ , K )P(γ ) (31)
where  is a normalization coefficient, which makes (31)
as a distribution. Through the (31), we combine the prior
probability and Kalman filter to convert the single-valued γ
(25) into a conditional distribution, which enhances the robustness
of the system and stabilizes the motion model.
We summarize the above method together. Table. III shows
the pseudo-code of our final algorithm.
(26)
V. E VALUATION
This section describes the experiment scenario, device
preparation, experiment processing, and result analysis. All
the tests are actual measurements.
Downloaded from https://iranpaper.ir
WANG et al.: SMARTPHONE-BASED PEDESTRIAN NLOS POSITIONING BASED ON ACOUSTICS AND IMU PARAMETER ESTIMATION
TABLE III
P SEUDO -C ODE OF O UR M ETHODS
Fig. 8. Different chirp triggering strategies by division time.
Fig. 9. Power supply of beacons. The integrated lithium battery inside,
so the width exceeds two coins.
A. Implementation
The proposed methods are evaluated by a Huawei Mate
9 with Android 9.0, Hisilicon Kirin 960 CPU, and 4 GB
memory as a middle-level configuration COTS phone. The
beacon consists of an amplified speaker, a power supply
module, a sync module, and an STM 32 chip (about five
dollars each). We modulate the chirp signal with a duration of
40 ms [11]. To ensure that most people are insensible to the
modulated chirp signals, we use a 4 kHz bandwidth, sweeping
from 18 kHz to 22 kHz, transmitting near-ultrasonic signals.
Xk+1 = Xk+1|k + Kk+1 [(1 − L k+1 )(zk+1|k − HXk+1|k ) + L k+1 Xk+1|k ]
= (1 − 
L k+1 )Kk+1 zk+1|k + [I − (1 − L k+1 )Kk+1 H + L k+1 Kk+1 ] Xk+1|k
 
BPE
Zk+1
Authorized licensed use limited to: Universitaet Bielefeld. Downloaded on November 25,2023 at 11:32:49 UTC from IEEE Xplore. Restrictions apply.
https://www.tarjomano.com https://www.tarjomano.com
23103
TABLE IV
N UMBER OF B EACONS AND P OSITION L OSS
R ATE (PLR) IN LOS AND NLOS
Fig. 10. Hold the phone and walk along the pre-prepared ground truth
line. The smartphone calculates real-time positioning coordinates.
The above limitation is that some commercial smartphones’
maximum sampling frequency is only 44.1 kHz or 48 kHz.
Although some Android smartphones are preset with 96 kHz
or higher, the smartphone’s hardware can only support up to
48 kHz. The sound pressure level (SPL) is less than 60 dB,
guaranteeing human safety. Under 60 dB SPL, the acoustic
signal can effectively propagate over 30 m, so we adjust the
time-division gap to 150 ms, as shown in Fig. 8.
Note that we can receive and process only one signal
for each interval when the beacons emit chirps succes-
sively. It avoids the classification of multi-signals, such as
multi-signal cocktail issues. We hold a smartphone in hand and
move according to a pre-defined route. Meanwhile, we also
evaluated the influence of the number of beacons on the
position loss rate under different indoor scenarios (LOS and
NLOS). The result is shown in Table. IV. When the number
of beacons is only four, nearly 30% of the signals may be lost
and cause tremendous errors. Beacons are usually fixed on
the walls and are powered either by batteries or 5V adapters,
as shown in Fig. 9.
B. Benchmark and Ground Truth
As shown in Fig. 10 (a), we stuck white tape on the floor
in advance as the ground truth, and the actor walked along the
white route. There are a few centimeters in error between the
trace of a phone and the ground truth line, which is difficult
to avoid.
C. Positioning Experiment of LOS
The experiment scenario set no artificial obstacles, and
all areas could theoretically be considered unobstructed. The
(30)
BPE
Xk+1
Downloaded from https://iranpaper.ir
https://www.tarjomano.com https://www.tarjomano.com
23104 IEEE SENSORS JOURNAL, VOL. 22, NO. 23, 1 DECEMBER 2022

Fig. 11. The test scenario and ground truth route. The illustrated route
can traverse every part of the room. All of the tables and chairs show no
more than 1 meter high.

Fig. 14. Comparison between KFLS and KFBPE for different numbers
of beacons.

Fig. 12. Comparison of the LOS results. The axis is room size.

Fig. 15. Probability density of error, mean μ, and standard deviation δ

comparison between KFLS, KFBPE and normal kalman.

Fig. 13. Error distribution in LOS. The maximum error in pure PDR is
too large and out of the bound.

beacons are uniformly distributed according to the optimal

Geometric Dilution Precision (GDOP). In this experiment, all
six Beacons were turned on. We defined it as an entire LOS
scenario, as shown in Fig. 11. Nevertheless, the actor will
inevitably block some signals behind himself during the walk.
It can be seen from Figs. 12 and 13 that acoustics, the KFLS,
and the KFBPE are similar in LOS performance. Fig. 16. The γ varies with the step number comparing with KFLS, KFLS
We got high accuracy positioning in this experiment, except after Moving Average (MA), and KFBPE. The red line of KFBPE shows
the expectation.
for drift points and outliers in several specific areas. Two
cabinet air-conditioners are fixed in the working model at the
left and right corners, affecting the corner temperature. Tem- in the 46 s walking, conforming to the theoretical value of
perature changes will inevitably lead to sound velocity changes cumulative error. Here, the step size of the PDR algorithm is
and affect localization. Especially in the right bottom corner, estimated as the Weinberg algorithm, the quaternion updates
the yellow line showed the outliers with severe multi-path the orientation, and the trajectory follows the Attitude and
fused temperature change. Due to body buffering, the accuracy Heading Reference System (AHRS) principle. The advantage
of the PDR algorithm has been decreased by more than 2 m of the proposed method did not fully reflect in the LOS

Authorized licensed use limited to: Universitaet Bielefeld. Downloaded on November 25,2023 at 11:32:49 UTC from IEEE Xplore. Restrictions apply.
Downloaded from https://iranpaper.ir
https://www.tarjomano.com https://www.tarjomano.com
WANG et al.: SMARTPHONE-BASED PEDESTRIAN NLOS POSITIONING BASED ON ACOUSTICS AND IMU PARAMETER ESTIMATION 23105

Fig. 17. Screen board position and ground truth route under two scenarios of NLOS. The blue circle highlights the screening board.

Fig. 18. Coordinate of S1 and S2, different amount of beacons, mixed gender.

scenario. Six beacons are fixed to achieve a 150% redundancy
rate, whereas minimum positioning requires four beacons.
Fig. 14 shows a different number of beacons required
for positioning accuracy. Both KFLS and KFBPE obtain a
superior accuracy, 4, 5, 6 represents the number of beacons.
We switched off the B5 and B6 beacons in Fig. 12 sequen-
tially. KFBPE will improve accuracy more than the KFLS
method, whereas different numbers of beacons have a slight
impact.
The probability density of error for our proposed method
and standard Kalman filter is shown in Fig. 15. KFBPE has a
concentrated density distribution with a small mean value and
strong robustness. All of them are distributed below 0.5 m,
of which 0.25 m error is the most concentrated. KFLS also has
a high probability distribution within 0.5 m, but its density is
relatively sparse. The error probability density of the standard
Kalman filter in the other paper approximately presents a
normal distribution with a mean value of 0.4 m, and its
performance improvement is slightly worse than that of the
above two methods. Fig. 16 shows the variation trend of γ in
KFLS and KFBPE. After moving averaging, the line of KFLS
serves as a contrast to show that even smoothing the γ in
KFLS is far less effective than KFBPE.
Measured by a large amount of previous data, the γ of
this user is 0.59, which is very close to the value evaluated
by KFBPE. Although KFLS fluctuates wildly, the average
is slightly close to 0.59. The experiment proves that both
methods effectively correct the personal coefficients γ based
on online self-learning, while KFBPE ensures low variance
and stability.
D. Positioning Experiment of NLOS
To further verify the robustness of the KFBPE method,
we set up a screen board occlusion experiment. The occlu-
sion environment has always been a tricky problem in
infrastructure-based localization. Most papers exert map con-
straints or floor matching methods to solve them. KFBPE
method uses IMU and fusion data effectively and is theoreti-
cally robust to occlusion.
Fig. 17 shows the two occlusion scenarios set in our experi-
ments. On the left is Scenario 1 (S1), in which the screen board
blocks the halfway aisle. Scenario 2 (S2) is on the right. The
screening board almost wholly cut off the lab, leaving only
a gap to walk. Due to the screen board barrier, we adjusted
the route to a loop for the S2. We notice that almost half
of the S2 suffers from regional signal loss and NLOS. The
floor is 2.7 m in height, and the screen panel is 2.65 m in
height. Acoustic signals can hardly pass through from above.
In contrast to the screen board, the office desk and seats
in the middle have a maximum height of 1 m, making up
a weak occlusion environment. The positioning accuracy in
such a scenario is more convincing and closer to the actual
application.
The results in Fig. 18 with both male and female users
show the positioning performance for two scenarios. The
gender difference has no discernible effect on positioning,
but the beacon number plays a significant role. It represents
a different amount and distribution of beacons, which will
result in different outliers and performance. Compared with
Fig. 19, although the positioning performance under the NLOS
condition is not as ideal as LOS, we still control the maximum

Authorized licensed use limited to: Universitaet Bielefeld. Downloaded on November 25,2023 at 11:32:49 UTC from IEEE Xplore. Restrictions apply.
Downloaded from https://iranpaper.ir
23106
Fig. 19. Cumulative error distribution comparison between different settings. The error in S1 is obviously lower than S2.
TABLE V
C OMPARISON OF P OSITIONING AND T IME -C OSTUMING R ESULTS IN S1, S2, AND LOS T ESTS (6 B EACONS )
Fig. 20. Box plot of the error distribution. The first six columns correspond
to Figs. 18(a) and 19(a) in half-occluded, and the last six columns
correspond to Figs. 18(b) and 19(b) in full-occluded.
error below 1 m, and the average error distribution is less
than 30 cm. We find approximate error performance compared
with S1 and S2, reflecting the strong robustness of the KFBPE
method from other viewings. The KFLS method is not suitable
for this scenario.
Due to the screen barrier, the testers cannot walk a circle
along the same way in S1 and S2. More than three directional
acoustic signals still exist in the left half of the laboratory and
the right half, maybe only remaining less than two beacon
signals. Without the KFBPE method, two signals can only
determine the fuzzy positioning, which means location failure.
At that time, the KFBPE method can obtain valid information
from the exact value on the left half, adjust the step coefficient,
and improve the right half’s positioning accuracy. Even if the
number of beacons decreased to 4, KFBPE still maintains a
stable track path without outliers.
Authorized licensed use limited to: Universitaet Bielefeld. Downloaded on November 25,2023 at 11:32:49 UTC from IEEE Xplore. Restrictions apply.
https://www.tarjomano.com https://www.tarjomano.com
IEEE SENSORS JOURNAL, VOL. 22, NO. 23, 1 DECEMBER 2022
Box plot, as Fig. 20 of positioning error more vividly
expresses the error distribution. The first six columns represent
six sets of experiments for S1, and the last six represent six
sets of experiments for S2. The median overall error is between
15 cm and 40 cm, whereas the lower quartile is mostly
within 20 cm. The maximum outlier is less than 1 m, and
the distribution is relatively flat. The sixth set, corresponding
to S1-6-actress, is one of the best data besides the maximum
outliers. Overall, the beacon quantity is positively correlated
with the positioning performance.
Table. V shows a result comparison of the methods pro-
posed in this paper. Due to the NLOS, the acoustics accuracy
of S1 and S2 is the worst, especially in the S2 scene. The
trace of PDR is not limited by occlusion but only by moving
distance, leading the similar results in the three scenarios. The
standard Kalman filter is better than acoustics and PDR but
not as good as KFLS and KFBPE. Among them, KFBPE
achieves the best positioning results without impacting the
NLOS of S1 and S2. The Longest Positioning Time (LPT) in
the last column of each scenario is in Table. V, both KFLS and
KFBPE algorithms can complete positioning calculation and
synchronous display within 1 s. Due to the loss of acoustics,
there is a situation where there is no positioning point for a
long time. Especially in the S2 scenario, the LPT is up to 12 s.
VI. C ONCLUSION
We propose a novel positioning system with strong robust-
ness in a complex environment. The core is Kalman Filter
and Bayesian parameter estimation so that (i) it fuses acoustic
and IMU data, (ii) users can localize their position only on a
smartphone, (iii) it can handle a variety of LOS and NLOS sce-
narios, (iv) it avoids getting user privacy. A Bayesian method
is applied to estimate the step coefficient, which solves the
Downloaded from https://iranpaper.ir
WANG et al.: SMARTPHONE-BASED PEDESTRIAN NLOS POSITIONING BASED ON ACOUSTICS AND IMU PARAMETER ESTIMATION
inaccurate positioning of hand-held smartphones, eliminates
IMU’s accumulated error, and figures out the customized para-
meter by online self-learning. We compare KFLS and KFBPE
methods to other methods, showing a significant positioning
accuracy, especially in some NLOS conditions. Future work
will leverage Long Short-Term Memory (LSTM) to analyze
acoustic signal characteristics under NLOS and combine the
clustering to deal with NLOS data further.
R EFERENCES
[1] R. C. Shit, S. Sharma, D. Puthal, and A. Y. Zomaya, “Location of things
(LoT): A review and taxonomy of sensors localization in IoT infrastruc-
ture,” IEEE Commun. Surveys Tuts., vol. 20, no. 3, pp. 2028–2061,
3rd Quart., 2018.
[2] M. Z. Win, W. Dai, Y. Shen, G. Chrisikos, and H. V. Poor, “Network
operation strategies for efficient localization and navigation,” Proc.
IEEE, vol. 106, no. 7, pp. 1224–1254, Jul. 2018.
[3] X. Wang, C. Zhang, F. Liu, and X. Xu, “Exponentially weighted particle
filter for simultaneous localization and mapping based on magnetic
field measurements,” IEEE Trans. Instrum. Meas., vol. 66, no. 7,
pp. 1658–1667, Jul. 2017.
[4] D. Lymberopoulos and J. Liu, “The Microsoft indoor localization com-
petition: Experiences and lessons learned,” IEEE Signal Process. Mag.,
vol. 34, no. 5, pp. 125–140, Sep. 2017.
[5] Microsoft. (2018). 2018competition. [Online]. Available:
https://www.microsoft.com/enus/research/event/microsoft-indoor-
localization-competition-ipsn-2018/
[6] F. Zafari, A. Gkelias, and K. K. Leung, “A survey of indoor localization
systems and technologies,” IEEE Commun. Surveys Tuts., vol. 21, no. 3,
pp. 2568–2599, 3rd Quart., 2017.
[7] V. Chamola, V. Hassija, V. Gupta, and M. Guizani, “A comprehensive
review of the COVID-19 pandemic and the role of IoT, drones, AI,
blockchain, and 5G in managing its impact,” IEEE Access, vol. 8,
pp. 90225–90265, 2020.
[8] E. Naserian, X. Wang, X. Xu, and Y. Dong, “A framework of loose
travelling companion discovery from human trajectories,” IEEE Trans.
Mobile Comput., vol. 17, no. 11, pp. 2497–2511, Nov. 2018.
[9] E. Naserian, X. Wang, K. Dahal, Z. Wang, and Z. Wang, “Person-
alized location prediction for group travellers from spatial–temporal
trajectories,” Future Gener. Comput. Syst., vol. 83, pp. 278–292,
Jun. 2018.
[10] C. Peng, G. Shen, and Y. Zhang, “BeepBeep: A high-accuracy acoustic-
based system for ranging and localization using COTS devices,” ACM
Trans. Embedded Comput. Syst., vol. 11, no. 1, pp. 1–29, 2012.
[11] L. Zhang, M. Chen, X. Wang, and Z. Wang, “TOA estimation of
chirp signal in dense multipath environment for low-cost acoustic
ranging,” IEEE Trans. Instrum. Meas., vol. 68, no. 2, pp. 355–367,
Feb. 2019.
[12] N. Rajagopal, P. Lazik, N. Pereira, S. Chayapathy, B. Sinopoli, and
A. Rowe, “Enhancing indoor smartphone location acquisition using floor
plans,” in Proc. 17th ACM/IEEE Int. Conf. Inf. Process. Sensor Netw.
(IPSN), Apr. 2018, pp. 278–289.
[13] H. Wang, L. Zhang, Z. Wang, and X. Luo, “Pals: High-accuracy pedes-
trian localization with fusion of smartphone acoustics and PDR,” in Proc.
IPIN, 2019, pp. 291–298.
[14] L. E. Díez, A. Bahillo, J. Otegui, and T. Otim, “Step length estimation
methods based on inertial sensors: A review,” IEEE Sensors J., vol. 18,
no. 17, pp. 6908–6926, Sep. 2018.
[15] J. Jahn, U. Batzer, J. Seitz, L. Patino-Studencka, and J. G. Boronat,
“Comparison and evaluation of acceleration based step length estimators
for handheld devices,” in Proc. Int. Conf. Indoor Positioning Indoor
Navigat., Sep. 2010, pp. 1–6.
[16] K.-H. Chow, H. He, J. Tan, and S.-H. G. Chan, “Efficient locality
classification for indoor fingerprint-based systems,” IEEE Trans. Mobile
Comput., vol. 18, no. 2, pp. 290–304, Feb. 2019.
[17] L. Sun, S. Sen, D. Koutsonikolas, and K.-H. Kim, “WiDraw: Enabling
hands-free drawing in the air on commodity WiFi devices,” in Proc. 21st
Annu. Int. Conf. Mobile Comput. Netw., Sep. 2015, pp. 77–89.
[18] J. Su, Z. Yao, and M. Lu, “An improved position determination algorithm
based on nonlinear compensation for ground-based positioning systems,”
IEEE Access, vol. 7, pp. 23675–23689, 2019.
Authorized licensed use limited to: Universitaet Bielefeld. Downloaded on November 25,2023 at 11:32:49 UTC from IEEE Xplore. Restrictions apply.
https://www.tarjomano.com https://www.tarjomano.com
23107
[19] F. Hoflinger, R. Zhang, P. Fehrenbach, J. Bordoy, L. Reindl, and
C. Schindelhauer, “Localization system based on handheld inertial sen-
sors and UWB,” in Proc. IEEE Int. Symp. Inertial Sensors Syst. (INER-
TIAL), Mar. 2017, pp. 1–2.
[20] Wikipedia. (2020). 2020airtag. [Online]. Available: https://en.wikipedia.
org/wiki/AirTags
[21] W. Hess, D. Kohler, H. Rapp, and D. Andor, “Real-time loop closure
in 2D LIDAR SLAM,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA),
May 2016, pp. 1271–1278.
[22] R. Zhang, W.-D. Zhong, K. Qian, and D. Wu, “Image sensor based
visible light positioning system with improved positioning algorithm,”
IEEE Access, vol. 5, pp. 6087–6094, 2017.
[23] J. Lee, “4–1: Invited paper: Mobile AR in your pocket with Google
Tango,” in SID Symp. Dig. Tech. Papers, 2017, vol. 48, no. 1, pp. 17–18.
[24] S. Yuan et al., “DSmT-based ultrasonic detection model for estimating
indoor environment contour,” IEEE Trans. Instrum. Meas., vol. 69, no. 7,
pp. 4002–4014, Jul. 2020.
[25] B. Zhou, M. Elbadry, R. Gao, and F. Ye, “BatTracker: High preci-
sion infrastructure-free mobile device tracking in indoor environments,”
in Proc. 15th ACM Conf. Embedded Netw. Sensor Syst., Nov. 2017,
p. 13.
[26] Y. Liu, W. Zhang, Y. Yang, W. Fang, F. Qin, and X. Dai, “RAMTEL:
Robust acoustic motion tracking using extreme learning machine for
smart cities,” IEEE Internet Things J., vol. 6, no. 5, pp. 7555–7569,
Oct. 2019.
[27] Z. Wang, H. Zhao, S. Qiu, and Q. Gao, “Stance-phase detec-
tion for ZUPT-aided foot-mounted pedestrian navigation system,”
IEEE/ASME Trans. Mechatronics, vol. 20, no. 6, pp. 3170–3181,
Dec. 2015.
[28] Q. Tian, Z. Salcic, K. Wang, and Y. Pan, “A multi-mode dead reckoning
system for pedestrian tracking using smartphones,” IEEE Sensors J.,
vol. 16, no. 7, pp. 2079–2093, Apr. 2016.
[29] Z. Xu, J. Wei, J. Zhu, and W. Yang, “A robust floor localization method
using inertial and barometer measurements,” in Proc. Int. Conf. Indoor
Positioning Indoor Navigat. (IPIN), Sep. 2017, pp. 1–8.
[30] D. Chen et al., “S2M: A lightweight acoustic fingerprints-based wireless
device authentication protocol,” IEEE Internet Things J., vol. 4, no. 1,
pp. 88–100, Feb. 2017.
[31] X. Xu, J. Yu, Y. Chen, Y. Zhu, and M. Li, “Leveraging acoustic signals
for vehicle steering tracking with smartphones,” IEEE Trans. Mobile
Comput., vol. 19, no. 4, pp. 865–879, Apr. 2020.
[32] H. Yang et al., “Smartphone-based indoor localization system using iner-
tial sensor and acoustic transmitter/receiver,” IEEE Sensors J., vol. 16,
no. 22, pp. 8051–8061, Nov. 2016.
[33] I. E. Radoi, D. Cirimpei, and V. Radu, “Localization systems reposi-
tory: A platform for open-source localization systems and datasets,” in
Proc. Int. Conf. Indoor Positioning Indoor Navigat. (IPIN), Sep. 2019,
pp. 1–8.
[34] W. Mao, J. He, and L. Qiu, “CAT: High-precision acoustic motion track-
ing,” in Proc. 22nd Annu. Int. Conf. Mobile Comput. Netw., Oct. 2016,
pp. 69–81.
[35] S. U. N. Wood, J. Rouat, S. Dupont, and G. Pironkov, “Blind
speech separation and enhancement with GCC-NMF,” IEEE/ACM
Trans. Audio Speech Lang. Process., vol. 25, no. 4, pp. 745–755,
Apr. 2017.
[36] K. C. Ho, “Bias reduction for an explicit solution of source local-
ization using TDOA,” IEEE Trans. Signal Process., vol. 60, no. 5,
pp. 2101–2114, May 2012.
[37] J. Nikolic, P. Furgale, A. Melzer, and R. Siegwart, “Maximum likelihood
identification of inertial sensor noise model parameters,” IEEE Sensors
J., vol. 16, no. 1, pp. 163–176, Jan. 2016.
[38] C. Tudor-Locke et al., “How fast is fast enough? Walking cadence
(steps/min) as a practical estimate of intensity in adults: A narrative
review,” Brit. J. Sports Med., vol. 52, no. 12, pp. 776–788, Jun. 2018.
[39] F. Gu, K. Khoshelham, C. Yu, and J. Shang, “Accurate step length esti-
mation for pedestrian dead reckoning localization using stacked autoen-
coders,” IEEE Trans. Instrum. Meas., vol. 68, no. 8, pp. 2705–2713,
Oct. 2018.
[40] H. Abdelnasser et al., “SemanticSLAM: Using environment landmarks
for unsupervised indoor localization,” IEEE Trans. Mobile Comput.,
vol. 15, no. 7, pp. 1770–1782, Jul. 2016.
[41] H. Weinberg, “Using the ADXL202 in pedometer and personal
navigation applications,” Analog Devices, Norwood, MA, USA,
Tech. Rep., AN-602, 2002, vol. 2, no. 2, pp. 1–6.
[42] V. Renaudin, M. Susi, and G. Lachapelle, “Step length estimation
using handheld inertial sensors,” Sensors, vol. 12, no. 7, pp. 8507–8525,
Jun. 2012.
Downloaded from https://iranpaper.ir
23108
Hucheng Wang (Student Member, IEEE)
received the B.Eng. degree in information
and communication technology in 2015. He is
currently pursuing the Ph.D. degree with the
School of Computer Science and Information
Security, Guilin University of Electronic
Technology, Guilin, China. He is a Visiting
Scholar with the College of Control Science and
Engineering, Zhejiang University, Hangzhou,
China. His research interests include indoor
localization, acoustic signal processing, and
data fusion.
Can Xue received the B.Sc. degree in automa-
tion from the Chongqing University of Posts
and Telecommunications, Chongqing, China,
in 2019. He is currently pursuing the Ph.D.
degree with the Department of Control Science
and Engineering, Zhejiang University, Hangzhou,
China. His current research interests include
indoor positioning, mobile systems, and acoustic
signal processing.
Zhi Wang (Member, IEEE) received the B.Eng.
degree from Shenyang Jianzhu University
in 1991, the M.Sc. degree from Southeast
University in 1997, and the Ph.D. degree
from the Shenyang Institute of Automation,
Chinese Academy of Sciences, in 2000.
From 2000 to 2002, he was a Postdoctoral
Fellow with the Institute National Polytechnique
de Lorraine, France; and Zhejiang University,
China. He is currently a Ph.D. Advisor with the
College of Control Science and Engineering,
Zhejiang University. His research areas include acoustic signal and array
signal processing, acoustic SLAM and robot positioning and navigation,
indoor localization and temporal-spatial data mining, device-free human
activity recognition, compressed sensing, deep learning, data fusion
of multiple sensing systems, Industrial Internet of Things (IIoT), and
privacy protection in crowdsourcing systems. He was also the winner of
the 2018 Microsoft Indoor Localization Competition in the Sound Group.
Authorized licensed use limited to: Universitaet Bielefeld. Downloaded on November 25,2023 at 11:32:49 UTC from IEEE Xplore. Restrictions apply.
https://www.tarjomano.com https://www.tarjomano.com
IEEE SENSORS JOURNAL, VOL. 22, NO. 23, 1 DECEMBER 2022
Lei Zhang (Student Member, IEEE) received
the B.Eng. and M.Sc. degrees in mechatron-
ics engineering from Chang’an University, Xi’an,
China, in 2007 and 2013, respectively, and the
Ph.D. degree in control science and engineer-
ing from Zhejiang University, Hangzhou, China,
in 2019. He is currently a Lecturer with the School
of Construction Machinery, Chang’an University.
His academic interests include machine learn-
ing, acoustic signal processing, time-frequency
analysis, indoor positioning, and tracking.
Xiaonan Luo (Senior Member, IEEE) was the
Director of the National Engineering Research
Center of Digital Life, Sun Yat-sen University,
Guangzhou, China. He is currently a Professor
with the School of Computer and Information
Security, Guilin University of Electronic Technol-
ogy, Guilin, China. His current research interests
include computer graphics, machine learning,
and pattern recognition. He received the National
Science Fund for Distinguished Young Schol-
ars granted by the National Natural Science
Foundation of China.
Xinheng Wang (Senior Member, IEEE) received
the B.Eng. and M.Sc. degrees in electrical engi-
neering from Xi’an Jiaotong University, Xi’an,
China, in 1991 and 1994, respectively, and the
Ph.D. degree in electronics and computer engi-
neering from Brunel University, Uxbridge, U.K.,
in 2001. He is currently a Professor with the
School of Advanced Technology and the Head
of the Department of Mechatronics and Robot-
ics, Xi’an Jiaotong-Liverpool University (XJTLU),
Suzhou, China. Prior to joining XJTLU, he was a
Professor with different universities in the U.K. He has been an Inves-
tigator or a Co-Investigator of nearly 30 research projects sponsored
from EU, U.K. EPSRC, Innovate U.K., China NSFC, and industry. He has
authored or coauthored over 170 referred papers. He holds 15 granted
patents, including one U.S., one Japan, four South Korea, and nine
China patents. His current research interests include tactile Internet,
indoor positioning, the Internet of Things (IoT), acoustic localization,
communications and sensing, and big data analytics for intelligent ser-
vices, where he has developed the world’s first smart trolley with Chigoo
Interactive Technology Company Ltd. His research has led to a few
commercial products in condition monitoring, wireless mesh networks,
and user-centric routing and navigation for group users.