Involving Ultra-Wideband in Consumer-level Devices into the

Ecosystem of Wireless Sensing

Junqi Ma†∗ , Zhaoxin Chang⋇† , Fusang Zhang†∗ , Jie Xiong‡ , Jiazhi Ni§ ,
Beihong Jin†∗ , Daqing Zhang⋇♯ ,
†
Institute of Software, Chinese Academy of Sciences, ∗ University of Chinese Academy of Sciences,
‡ University of Massachusetts Amherst, § Tencent Inc. ⋇ Institut Polytechnique de Paris, France ♯ Peking University,

ABSTRACT
Among various wireless sensing modalities, Ultra-Wideband (UWB) Smart TV

exhibits unique advantages such as fine granularity owing to its UWB

super large bandwidth (500 MHz – 2 GHz). Though promising, connection Smart speaker

UWB sensing was only demonstrated on dedicated hardware in- Smart speaker

cluding DW1000 and XETHRU X4 which are not available in exist-
ing consumer-level devices. In the last few years, we observed an
interesting trend of UWB module being embedded into consumer-
level devices such as smartphones and smart watches. However,
leveraging UWB module inside consumer-level devices for sensing
poses new challenges. One key challenge is that while dedicated
UWB hardware can present us with raw physical-layer signal am-
plitude and phase, only upper-layer distance and angle information
can be extracted from consumer-level devices. In this demo, we
address the challenges and present the first UWB sensing system
hosted on iPhone and Apple Watch without any dedicated hard-
ware components. We show that with just the upper-layer UWB
data reported from smartphones, exciting sensing applications such
as fine-grained 3D handwriting and multi-target tracking can be
realized, pushing RF sensing one step forward towards real-life
adoption.
CCS CONCEPTS
• Human-centered computing → Ubiquitous and mobile com-
puting systems and tools.
1 INTRODUCTION
Recent years have witnessed rapid development of wireless sens-
ing using different RF technologies including WiFi [9], RFID [6],
LoRa [10] and UWB [4]. A large variety of sensing applications have
been realized including human tracking [7], gesture recognition [8],
vital sign monitoring [11] and even environment sensing [3]. Al-
though promising, these sensing systems still require dedicated
hardware. For example, WiFi sensing usually adopts Intel 5300
WiFi cards and UWB sensing employs the DW1000 transceiver IC.
These hardware components are not used in any consumer-level
electronics such as our smartphones or home appliances.
In the last few years, we observed an exciting opportunity to
realize RF sensing on consumer-level electronics (e.g., smartphones)
Permission to make digital or hard copies of part or all of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
ACM MobiCom ’22, October 17–21, 2022, Sydney, NSW, Australia
© 2022 Copyright held by the owner/author(s).
ACM ISBN 978-1-4503-9181-8/22/10.
https://doi.org/10.1145/3495243.3558745
UWB connection
Smart watch UWB
connection
(a) Handwriting recognition (b) Human trajectory tracking
Figure 1: UWB based sensing applications.
for the first time, i.e., more and more consumer-level electronics
are now equipped with Ultra-Wideband (UWB) chips. Since 2019,
UWB has been a default component in iPhone. Apple continued
this trend to add UWB chips to Apple Watch, HomePod mini and
AirTag. Following this trend, Samsung, Xiaomi and Google also
include UWB chips in their latest products, including Galaxy Note
20, Note 20 Ultra, Xiaomi MIX 4, Xiaomi smart speaker, Mi TV, and
Pixel 6 Pro.
While promising, we quickly found one challenge when we adopt
consumer-level UWB-equipped devices such as iPhone for sensing.
Existing UWB sensing systems can report raw physical-layer sig-
nal samples containing amplitude and phase information [5, 12].
The signal amplitude/phase variations are utilized to sense tar-
get’s information. However, consumer-level devices can only report
upper-layer distance and angle information [1, 2]. So the obvious
question is: can the distance and angle information be utilized for
fine-grained sensing? If so, what type of sensing applications can
be supported? To answer these questions, we employ two typical
sensing applications, i.e., handwriting recognition and multi-target
tracking as shown in Figure 1 which require different levels of
sensing granularity to showcase the capability of UWB sensing on
consumer-level devices.
1) Handwriting recognition. A lot of handwriting recognition
systems have been proposed. However, we want to emphasize the
unique advantages of our system: (i) with just one single smart-
phone, we can track a smart watch for handwriting recognition
in 3-dimensional space; (ii) our system can achieve a relatively
large sensing range (3–4 m); and (iii) the proposed system is not
constrained by target orientation which is a common problem for
RF sensing.
2) Multi-target tracking. In this application, we show that with
just one UWB device serving as the anchor, the proposed system

758
 UWB data collection Handwriting recognition
Recognition
Sensing scenarios Azimuth angle Motion direction Feature Continues word result
identification extraction recognition
In-air Smart
‘Hello’
handwriting Smartphone
watch
Elevation angle
Multi-target tracking
Human UWB
UWB device Data Trajectory User1
tracking connection Distance
selection fusion smoothing User2

Figure 2: Consumer-level UWB based Sensing System.

can simultaneously track the moving trajectories of multiple human

targets wearing UWB-equipped devices in real-time without a need Handwriting trajectory
of deploying any other infrastructure.
We implemented our system on iPhone and Apple Watch. We
propose new signal processing schemes to make wireless sensing
happen on consumer-level UWB electronics. We believe this work
Apple
pushes RF sensing from dedicated hardware-based lab prototypes Watch iPhone
to consumer-level electronics for the first time, moving RF sensing
one step towards real-life adoption. (a) Demo scenario (b) Handwriting words

2 SYSTEM DESIGN Figure 3: Demo 1: Handwriting recognition.

As shown in Figure 2, the proposed system includes three parts, i.e.,
UWB data collection and processing, handwriting recognition and
multi-target tracking. Multi-target tracking. The demo scenario of multi-target track-
UWB data collection. We develop an App based on Apple’s ing is shown in Figure 4. We place an iPhone 12 Pro Max on the
Nearby Interaction API [1] to collect UWB data from iPhone and tripod to serve as an anchor. There are a total of three human tar-
Apple Watch. Nearby Interaction API supports peer-to-peer UWB gets walking in a hall with a size of 13𝑚 × 6𝑚. Among the three
connection between devices (e.g., iPhone-to-iPhone and iPhone- targets, two of them have an iPhone 11 in their pockets and the
to-Apple Watch). For example, after the iPhone-to-Apple Watch last one wears an Apple Watch Series 6. The three targets’ walking
pairing is successful, iPhone can continually report the distance, trajectories can be highly accurately tracked in real-time with an
elevation angle, and azimuth angle of the Apple Watch. error below 10 cm. The tracking range can be up to 15 m and the
Handwriting recognition. To make 3D tracking work, we first current system supports up to three targets.
perform motion direction identification to find the writing plane,
and then project the 3D trajectory onto the identified 2D plane.
Then, we adopt a Convolutional Recurrent Neural Network (CRNN)
to perform word-level handwriting recognition.
Multi-target tracking. The proposed system can track targets
with just one single device serving as the anchor. The anchor device
broadcasts tokens to connected devices and each connected device is
assigned a unique sequence identifier which helps avoid collisions.

3 DEMO SETUPS
Our demos are composed of two applications, i.e., handwriting
recognition and multi-target tracking.
Handwriting recognition. Figure 3a shows the demo scenario
of handwriting recognition. We place an iPhone 12 Pro Max1 on
the table. The user is asked to wear an Apple Watch and sit in front Figure 4: Demo 2: Multi-target tracking
of the phone. When the user performs handwriting, the real-time
trajectory and recognition result are shown on the screen. Figure 3b
showcases the handwriting of words with variable lengths (2–7 ACKNOWLEDGMENTS
letters), including ‘ok’, ‘eye’, ‘look’, ‘hello’, ‘legend’ and ‘healthy’.
This work is partially supported by the National Natural Science
The user can freely change the location and orientation during the
Foundation of China (No. 62172394, No. 62072450) and the Youth
writing process. The user can also write in three dimensions.
Innovation Promotion Association, Chinese Academy of Sciences
1 The proposed system works on iPhone 11, 12 and 13. (No. 2020109).

759
REFERENCES
[1] 2021. Nearby Interaction. https://developer.apple.com/documentation/
nearbyinteraction
[2] 2022. Ultra-wide band communication. https://developer.android.com/guide/
topics/connectivity/uwb
[3] Zhaoxin Chang, Fusang Zhang, Jie Xiong, Junqi Ma, Beihong Jin, and Daqing
Zhang. 2022. Sensor-free Soil Moisture Sensing Using LoRa Signals. Proceedings
of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 2 (2022),
1–27.
[4] Weiyan Chen, Fusang Zhang, Tao Gu, Kexing Zhou, Zixuan Huo, and Daqing
Zhang. 2021. Constructing Floor Plan through Smoke Using Ultra Wideband
Radar. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous
Technologies 5, 4 (2021), 1–29.
[5] Zhe Chen, Tianyue Zheng, Chao Cai, and Jun Luo. 2021. MoVi-Fi: Motion-Robust
Vital Signs Waveform Recovery via Deep Interpreted RF Sensing. 392–405.
[6] Chao Feng, Jie Xiong, Liqiong Chang, Fuwei Wang, Ju Wang, and Dingyi Fang.
2021. RF-Identity: Non-Intrusive Person Identification Based on Commodity
RFID Devices. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5, 1, Article
9 (mar 2021), 23 pages. https://doi.org/10.1145/3448101
[7] Kai Niu, Xuanzhi Wang, Fusang Zhang, Rong Zheng, Zhiyun Yao, and Daqing
Zhang. 2022. Rethinking Doppler effect for accurate velocity estimation with
760
commodity WiFi devices. IEEE Journal on Selected Areas in Communications
(2022).
[8] Kai Niu, Fusang Zhang, Xuanzhi Wang, Qin Lv, Haitong Luo, and Daqing Zhang.
2021. Understanding WiFi Signal Frequency Features for Position-Independent
Gesture Sensing. IEEE Transactions on Mobile Computing (2021), 1–1. https:
//doi.org/10.1109/TMC.2021.3063135
[9] Youwei Zeng, Dan Wu, Jie Xiong, Jinyi Liu, Zhaopeng Liu, and Daqing Zhang.
2020. MultiSense: Enabling multi-person respiration sensing with commodity
wifi. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous
Technologies 4, 3 (2020), 1–29.
[10] Fusang Zhang, Zhaoxin Chang, Kai Niu, Jie Xiong, Beihong Jin, Qin Lv, and
Daqing Zhang. 2020. Exploring lora for long-range through-wall sensing. Pro-
ceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
4, 2 (2020), 1–27.
[11] Fusang Zhang, Zhaoxin Chang, Jie Xiong, Rong Zheng, Junqi Ma, Kai Niu, Bei-
hong Jin, and Daqing Zhang. 2021. Unlocking the beamforming potential of
lora for long-range multi-target respiration sensing. Proceedings of the ACM on
Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 2 (2021), 1–25.
[12] Xuezhao Zheng, Baoyuan Wang, and Ju Zhao. 2019. High-precision positioning
of mine personnel based on wireless pulse technology. PLOS ONE 14, 7 (07 2019),
1–25.