Synchronized acquisition system

based on BLE sensor network

for real-time IoT applications

Andrea Pignata

System-on-Chip Architectures - 15/11/22

1
System Flow

Board Firmware Python Client

Post-Processing
- Send sensor data - Gather sensors
at predefined BLE data .csv
- MATLAB tools
intervals and save to file
- Axes calibration
- Time - Time
synchronization synchronization

2
Used boards
ST SensorTile.box
- ARM Cortex-M4
microcontroller (STM32)
- 6-axis IMU (acceleromenter
+ gyroscope)
- 3-axis magnetometer
- Temperature, Humidity and
Pressure sensors
- Microphone (FFT)
ST BlueTile MetaMotionS
- ARM Cortex-M0 - ARM Cortex-M4
microcontroller microcontroller (Nordic)
- 6-axis IMU (acceleromenter - 6-axis IMU (acceleromenter
+ gyroscope) + gyroscope)
- 3-axis magnetometer - 3-axis magnetometer
- Temperature, Humidity and - Temperature and pressure
Pressure sensors sensors
- Microphone (BlueVoice) 3
Used boards
ST SensorTile.box ST BlueTile MetaMotionS

- ARM Cortex-M4 - ARM Cortex-M0 - ARM Cortex-M4

microcontroller (STM32) microcontroller microcontroller (Nordic)
CLOSED SOURCE FIRMWARE
OPEN SOURCE FIRMWARE - IMU
Immutable FW
- 6-axis IMU
- (acceleromenter - 6-axisbyIMU
Firmware can be modified the(acceleromenter
user - 6-axis (acceleromenter
+ gyroscope) + gyroscope) +- gyroscope)
Client can be developed
- Client development more
- 3-axis magnetometer
complex: you need
- 3-axis magnetometer withmagnetometer
- 3-axis manufacturer APIs
to and
- Temperature, Humidity start from- Temperature,
scratch Humidity and - Temperatureonly!
and pressure
Pressure sensors Pressure sensors sensors
- Microphone (FFT) - Microphone (BlueVoice) 4
Outline
• Introduction

• Bluetooth Low Energy overview

• Data streaming firmware

• Audio streaming firmware

• Client Python script

• MATLAB post-processing

5
Bluetooth Low Energy (BLE)
• Low-power, low-throughput
connection protocol
(2.4 GHz band) client server

• Server - Client architecture

Connection request
• Data exchange is based on
write, read operations
performed on the
GATT DB description
GATT database.

6
BLE: GATT Database
• Data stored in arrays
• 8-bit unsigned int
• Variable size Char 00e0 Acc, Gyr, Mag
(IMU) 🔔
• Arrays are called characteristics Service 0001 Char 001c Temp, Hum,
and are identified by UUIDs (sensors) (ENV) 🔔 Press
Char 0004 Dominant
• Characteristics are grouped into
(AUD) 🔔 Frequency
services
Service 000f Char 0002 Timestamp
• Characteristics have properties (time_sync) (TS) ✏️🔔
defining what can be done on
its array (write, read, notify…)

7
BLE: Read
• Read operation is the simplest one.

• The client sends a read request, client server

along with the characteristic UUID
and the server answers only one time
with the corresponding array value Read request

• Slow and not efficient for high-data rate

applications. Characteristic value

8
BLE: Notify
• Notifications system removes the need
for continuous read requests.
client server
• The client sends a notification
subscription,
along with the characteristic UUID. Notification subscription

• The server sends the array value every

time it is modified.

• Fastest way to send data via BLE. Characteristic value(s)

9
BLE: Write
• The client can send data to the server.

• The client sends a write request,

along with the characteristic UUID client server
and the value to be written.

Interrupt execution
Write request
• The server will launch an interrupt
procedure to deal with the write
request.

• Depending on the characteristic

properties, a confirmation may or may (possible) answer
not be sent.

10
How to deal with BLE
• nRF Connect
• A nice GUI to show BLE devices,
connect to them,
have access to read, write,
notifications…
• Bluetoothctl
• Command line tool for total
Bluetooth control on Linux
• Full support of GATT
characteristics in the GATT menu.
• BluePy
• Python library
• Includes functions for
• Connection
• Notification subscription
• Read/Write operations
• Complete documentation on
https://ianharvey.github.io/bluepy-
doc/
11
Outline
• Introduction

• Bluetooth Low Energy overview

• Data streaming firmware

• Audio streaming firmware

• Client Python script

• MATLAB post-processing

12
Firmware description
• A main procedure composed of:
• Configuration block: configures the microcontroller and the sensors (only one time)
• Endless loop:
• Check the status of different flags (if the sensor data ready)
• If flag == 1, launch the procedure that updates the corresponding BLE characteristic.

• Interrupt Service Routines, that can modify the values of those flags:
• Timers: set up at specific values, to send IMU and ENV sensor data at specific rates:
• 75Hz for IMU
• 1Hz for ENV (…humidity and temperature won’t vary at high rates…)
• DMA: responsible of microphone data conversion.
• BLE data received: the client can send data for time synchronization purposes, and the
interrupt takes care of this situation.
13
SensorTile.box flowchart

14
BlueTile flowchart

15
Time Synchronization
Time synchronization
• Re-synchronization moments
• Packets exchange between client and
node
• Alignment of timestamps
Example: BLE configuration

- aci_gatt are functions used to modify the GATT database

- Their explanations are available in the headers .h files and in the documentation
- Use the IDE tools (i.e. right click – Open Declaration) to explore the source code and find
header files quickly and easily
17
Example: IMU data send
- An array of uint_8 is created, it will be the
characteristic value.
- STORE_LE_16 is a function that splits 16-bit
values and saves them in 8-bit arrays, like our
buff. LE stands for Little Endian!
It needs the destination buffer and the
position to save data in.
- Then, the characteristic is updated via the
aci_gatt procedure.

- At the end, the characteristic will be an array

like this:

18
Example: Interrupt

- This is the Interrupt Procedure for Write Requests on the Time Synchronization char.
- The current timestamp of the sensor board is split in 8 bit array.
- The number received from the client is incremented by 1
- Timestamp and incremented value are sent back with the aci_gatt function.
- Use the IDE tools (i.e. Search) to find this procedure quickly and easily, as it is hidden!
- TIP: you can find the configuration for the Timers Interrupt in the InitTimers() function.
19
Outline
• Introduction

• Bluetooth Low Energy overview

• Data streaming firmware

• Audio streaming firmware

• Client Python script

• MATLAB post-processing

20
BlueVoice flowchart

21
Firmware description
• A main procedure composed of:
• Configuration block: configures the microcontroller and the sensors (only one time)
In this firmware, also the BlueVoice is initialized here.
• Endless loop:
• Make the BlueVoice FSM proceed.

• BlueVoice Interrupt Routines, that make the audio stream work correctly.

- Documentation about BlueVoice and the available functions:

bluevoice_adpcm_bnrg1.c
- In this firmware, the timestamp_helper.c file contains some functions, called inside
bluevoice_adpcm_bnrg1.c, that are sending out the timestamp every time audio data is sent.
These are not BlueVoice ufficial functions, and can be removed in case of problems or unstable
audio stream.
22
Outline
• Introduction

• Bluetooth Low Energy overview

• Data streaming firmware

• Audio streaming firmware

• Client Python script

• MATLAB post-processing

23
Clients description
• A client for ST devices based on BluePy library and composed of:
• Configuration block: connects and subscribes to notifications
• Endless loop: waits for notifications.
• 1s timer: regularly sends time synchronization write requests
• Delegate objects: receive notifications and write data to file
• CtrlC signal: stops data acquisition as the user presses Ctrl+C

• A client for MetaMotion devices, based on MBIENTLAB APIs:

• Configuration block: connects, configures sensors and subscribes to notifications
• Endless loop: waits for notifications.
• State objects: receive notifications and write data to file
• CtrlC signal: stops data acquisition as the user presses Ctrl+C
• User have no access on how these functions work.
24
Outline
• Introduction

• Bluetooth Low Energy overview

• Data streaming firmware

• Audio streaming firmware

• Client Python script

• MATLAB post-processing

25
Import scripts description
Available scripts:
• data_extract_st.m and data_extract_meta.m
• import data from csv files (generated respectively by ST client or META client)
• for gyro/acc/mag only!
• plot them independently
• transform them into timeseries structure (assign precise timestamp to each sample)
• reorganize timeseries (delete repeated entries, order them in time)
• Tip: data can now be seen in the Signal Analyzer Tool from the Digital Signal Processing
Toolbox.

• gyro_extract_st.m and gyro_extract_meta.m

• exactly the same script, but with different variable names.

26
Import scripts description
Available scripts:
• data_extract_frequency.m
• Generates timeseries variables starting from frequency csv files

• data_extract_voice.m
• Generates timeseries variables starting from BlueVoice files
• Works with the modified firmware only: it relies on the customized timesync mechanism

27
Calibration scripts description

28
Calibration script description

29
Calibration script description
Starting point of the script: untitled.m

• Initialize all, transform timetables into lists of samples.

Start = 0, Stop = last sample.  start/stop will represent the range of samples to consider.
• While start < stop
• is_the_AM35_static()
• fill_static_buffer(): fill the buffer of all the samples, from the start one, that
correspond to the static case.
Variance of the buffer (acc + gyro) is calculated and compared to a threshold.
• static_case(): calculate the alpha/gamma angles for rotation
• is_the_AM35_dynamic()
• Does the same, but for the dynamic case  only gyroscope is considered
• Beta angle is calculated.
30
Demonstration: video
Starting point of the script: untitled.m

• Initialize all, transform timetables into lists of samples.

Start = 0, Stop = last sample.  start/stop will represent the range of samples to consider.
• While start < stop
• is_the_AM35_static()
• fill_static_buffer(): fill the buffer of all the samples, from the start one, that
correspond to the static case.
Variance of the buffer (acc + gyro) is calculated and compared to a threshold.
• static_case(): calculate the alpha/gamma angles for rotation
• is_the_AM35_dynamic()
• Does the same, but for the dynamic case  only gyroscope is considered
• Beta angle is calculated.
31