Simulate and Deploy UAV Applications with

SIL and HIL Workflows

Mihir Acharya, MathWorks Ronal George, MathWorks Julia Antoniou, MathWorks

1
Agenda
Introduction

PX4 with Simulink

MIL, SIL and HIL Workflows

Scenario Simulation

HIL with Scenario Simulation

Summary and Resources

2
Fly a Drone Through a City Block

3
Simulation in a Virtual Scenario

Plan Mission Design & Simulate Validate & Deploy

4
Building Blocks for UAV Simulation

Plan Mission Design & Simulate Validate & Deploy

Design flight controller and simulate Deploy flight controller and autonomy
Generate flight path for the mission plant behavior in virtual scenarios algorithms to the platform

Simulated Flight Controller Plant Model

Waypoints

Ground Control Station

Scenario Simulation

5
MIL, SIL and HIL Workflows for UAV Simulation
Inputs Flight Controller Plant Model
Host Host
PC PC

Ground Control Station

Model-in-the-Loop
(MIL)

C++ EXE on Host PC Deploy on Hardware

Host
PC

Simulator communicates Simulator communicates

with the host PC with the hardware

Software-in-the-Loop Hardware-in-the-Loop
(SIL) (HIL) 6
HIL with Flight Controller Deployed on PX4

TCP/IP or UDP

Deploy on Hardware Deploy on Hardware Full HIL Workflow

PX4 HIL

• Communicates with
• PX4 sensors are disabled actuators, sensors and the
drone peripherals
• Psuedo real-time operation
• Real-time operation

Focus of this talk

7
 Visualize Flight Behavior in a Virtual Scenario
Inputs Flight Controller Plant Model Scenario Simulation
Host Host Host
PC PC PC

Ground Control Station

Model-in-the-Loop
(MIL)

C++ EXE on Host PC Deploy on Hardware

Host
PC

Software-in-the-Loop Hardware-in-the-Loop
(SIL) (HIL)
8
 Design Autonomy Algorithms with UAV Simulation Workflow
Inputs
Flight Controller Plant Model Scenario Simulation Onboard Autonomy
Host Host Host Host
PC PC PC PC

Ground Control Station

Model-in-the-Loop
(MIL)

C++ EXE on Host PC Deploy on Hardware Deploy on Hardware

Host
PC

NVIDIA® Jetson™

Software-in-the-Loop Hardware-in-the-Loop
9
What is Pixhawk®
Host machine

Power Management Board

On-board sensors:
• Accel/Gyro: ICM-20689
• Accel/Gyro: BMI055 or
ICM20602 USB Type-B
• Magnetometer: IST8310
Transmitter
• Barometer: MS5611
Interfaces:
• 8-16 PWM outputs
• 3 dedicated PWM
• Dedicated R/C Telemetry Link
• 5 serial ports
• 3 I2C ports
• 4 SPI buses
• 2 CANBuses
GPS module
Motor Output

https://docs.px4.io/en/assembly/quick_start_pixhawk4.html
10
UAV Toolbox Support Package for PX4 Autopilots
Simulink Block Library

uORB: Asynchronous pub/sub

messaging API (middleware)

11
PX4 Autopilots Support Package Relationship to PX4 Architecture

https://www.mathworks.com/help/supportpkg/px4/ug/px4-capabilities-integration.html
12
PX4 Autopilots Support Package Relationship to PX4 Architecture

Deploy Simulink Controller

https://www.mathworks.com/help/supportpkg/px4/ug/px4-capabilities-integration.html
13
 UAV Simulation Workflow with PX4 and Simulink
Inputs
Flight Controller Plant Model Scenario Simulation Onboard Autonomy
Host Host Host Host
PC PC PC PC

Ground Control Station

Model-in-the-Loop
(MIL)

C++ EXE on Host PC Deploy on Hardware Deploy on Hardware

Host
PC

NVIDIA Jetson

Software-in-the-Loop Hardware-in-the-Loop
14
Model In the Loop

Inputs Flight Controller Plant Model

Host Host
PC PC

Simulated Waypoints

Model-in-the-Loop
(MIL)

15
Quadcopter Simulation in Simulink

Shipping example in UAV Toolbox

16
Quadcopter Simulation in Simulink

Video included with session content package

17
Simulink Plant Modelling

High-Fidelity Low-Fidelity
Building UAV Programming UAV

18
Software-in-the-Loop (SIL)

Inputs
Flight Controller Plant Model
Host Host
PC PC

Simulated Waypoints

C++ EXE on Host PC

Host
PC

Software-in-the-Loop

20
 SIL with PX4 HSP and Simulink Plant

Flight Controller in SIL

Shipping example in UAV Toolbox

Plant Model in Simulink (See Task 2 in the example)

21
SIL with PX4 HSP and Simulink Plant

Video included with session content package

22
Hardware-in-the-Loop (HIL)

Inputs
Flight Controller Plant Model
Host Host
PC PC

Ground Control Station

Deploy on Hardware

Hardware-in-the-Loop
24
HIL with PX4 HSP and Simulink Plant

Video included with session content package

26
UAV Simulation Workflow with PX4 and Simulink
Inputs Flight Controller Plant Model
Host Host
PC PC

Ground Control Station

Model-in-the-Loop
(MIL)

C++ EXE on Host PC Deploy on Hardware

Host
PC

Simulator communicates Simulator communicates

with the host PC with the hardware

Software-in-the-Loop Hardware-in-the-Loop
(SIL) (HIL) 27
 UAV Simulation Workflow with PX4 and Simulink
Inputs
Flight Controller Plant Model Scenario Simulation Onboard Autonomy
Host Host Host Host
PC PC PC PC

Ground Control Station

Model-in-the-Loop
(MIL)

C++ EXE on Host PC Deploy on Hardware Deploy on Hardware

Host
PC

NVIDIA Jetson

Software-in-the-Loop Hardware-in-the-Loop
28
Closed-loop autonomy simulation

Scenario Simulation

Cuboid Photorealistic
Environments Environments

29
Create environments and simulate sensor readings
UAV Scenario Designer

30
Integrate environments into full-system simulation

31
Integrate environments into full-system simulation

32
Integrate environments into full-system simulation

33
 UAV Simulation Workflow with PX4 and Simulink
Inputs
Flight Controller Plant Model Scenario Simulation Onboard Autonomy
Host Host Host Host
PC PC PC PC

Ground Control Station

Model-in-the-Loop
(MIL)

C++ EXE on Host PC Deploy on Hardware Deploy on Hardware

Host
PC

NVIDIA Jetson

Software-in-the-Loop Hardware-in-the-Loop
34
HIL with Scenario Visualization

Inputs
Flight Controller Plant Model Scenario Simulation
Host Host Host
PC PC PC

Ground Control Station

Deploy on Hardware

Hardware-in-the-Loop
35
HIL with Scenario Visualization

Shipping example in UAV Toolbox

36
HIL with Scenario Visualization

38
 UAV Simulation Workflow with PX4 and Simulink
Inputs
Flight Controller Plant Model Scenario Simulation Onboard Autonomy
Host Host Host Host
PC PC PC PC

Ground Control Station

Model-in-the-Loop
(MIL)

C++ EXE on Host PC Deploy on Hardware Deploy on Hardware

Host
PC

NVIDIA Jetson

Software-in-the-Loop Hardware-in-the-Loop
Shipping example in UAV Toolbox 40
Simulate and Deploy UAV Applications with SIL and HIL Workflows

▪ Why SIL and HIL?

– Ensure safety in real flights
– Test flight behavior in simulation

▪ Why MATLAB & Simulink?

– Integrate with external autopilots
– Generate C/C++ code for onboard computers
Relevant MATLAB EXPO Workshop
– Scenario simulation with MATLAB and Unreal Engine

41
UAV Toolbox
UAV Algorithms for Scenario Design & Low-Fidelity Flight Telemetry Data Analysis
Planning and Control Sensor Simulation Flight Log Analyzer App

Connectivity and Deployment with Unreal Engine Co-Simulation with Reference Applications
MAVLink and PX4 Sensor Models
Board Deployment

42
Resources

▪ Product Page
– www.mathworks.com/products/uav.html

▪ Product Overview Video

– https://www.mathworks.com/videos/what-is-uav-toolbox-1600154005892.html

▪ UAV Toolbox Support Package for PX4 Autopilots

– https://www.mathworks.com/help/supportpkg/px4/index.html?s_tid=CRUX_topnav

▪ Documentation
– www.mathworks.com/help/uav/

▪ Examples
– www.mathworks.com/help/uav/examples.html
43
 Share the EXPO experience
#MATLABEXPO

Mihir Acharya, MathWorks Ronal George, MathWorks Julia Antoniou, MathWorks

https://www.linkedin.com/in/amihir/ https://www.linkedin.com/in/ronalgeorge/
https://www.linkedin.com/in/ronalgeorge/ https://www.linkedin.com/in/julia-antoniou/

44
Thank you

© 2022 The MathWorks, Inc. MATLAB and Simulink are registered trademarks of The MathWorks, Inc. See mathworks.com/trademarks
for a list of additional trademarks. Other product or brand names may be trademarks or registered trademarks of their respective holders.

45