Autonomous

Navigation, Object
Detection and 3D
Mapping on UAV
Matthew Strong
Brian Nguyen
Ayush Khanal
Rami Mouro
 Motivation: Groundbreaking Tech

● Groundbreaking
technology
● So much potential and
many uses
○ video and image
capturing, inspection,
surveillance,
recreational purposes,
etc.
 Motivation: Emerging Tech

● Work needs to be done in

order to have them be
more integrated into our
lives
 Motivation: Object Detection and 3D Mapping

● Ongoing pursuits
● Challenging and exciting to incorporate them into UAVs
 Tensorflow and PyTorch
● Used the SSD (Single Shot Detector) 300 model for PyTorch object detection
○ Works on 300 by 300 images
● Also used the pretrained model for object detection from NVIDIA’s
implementation
● Implemented for TensorRT with torch2trt
● Creates optimized inference engine from variety of frameworks such as PyTorch
and TF … way faster!
● Tensorflow + Keras = Dronet work
● All runs real-time
Dronet

● Double headed CNN

● Outputs steering angle and collision
probability
● Commands are sent to cmd_vel topic
● Udacity and custom dataset
● Integrated open-source code at
https://github.com/uzh-rpg/rpg_public_dr
onet/
● Runs locally on GPU
● Lots of work to adapt for ROS Kinetic
● GPU Errors
ROS and MavROS
 SLAM (Simultaneous Localization and Mapping)

● Explored various open-

source SLAM algorithms
○ ORB-SLAM
○ DSO
○ SVO
○ LSD-SLAM
Orb-Slam 2
DSO
SVO
LSD-SLAM
Demo
Sources
[Monocular] Raúl Mur-Artal, J. M. M. Montiel and Juan D. Tardós. ORB-SLAM: A Versatile
and Accurate Monocular SLAM System. IEEE Transactions on Robotics, vol. 31, no. 5, pp.
1147-1163, 2015. (2015 IEEE Transactions on Robotics Best Paper Award)..

[Stereo and RGB-D] Raúl Mur-Artal and Juan D. Tardós. ORB-SLAM2: an Open-Source
SLAM System for Monocular, Stereo and RGB-D Cameras. IEEE Transactions on Robotics,
vol. 33, no. 5, pp. 1255-1262, 2017..

[DBoW2 Place Recognizer] Dorian Gálvez-López and Juan D. Tardós. Bags of Binary
Words for Fast Place Recognition in Image Sequences. IEEE Transactions on Robotics, vol.
28, no. 5, pp. 1188-1197, 2012.

Loquercio, A., Maqueda, A. I., Del-Blanco, C. R., & Scaramuzza, D. (2018). DroNet:
Learning to Fly by Driving. IEEE Robotics and Automation Letters.
https://doi.org/10.1109/LRA.2018.2795643

Engel, J., Koltun, V., & Cremers, D. (2016). Direct Sparse Odometry. IEEE Transactions on Pattern Analysis
and Machine Intelligence, 40, 611-625.