0 ETH Lecture Introduction and Lecture Overview 2019

ASL
Autonomous Systems Lab
Spring 2019
Mobile Robots | Introduction and Lecture Overview

Autonomous Mobile Robots
https://online.ethz.ch/courses/course-v1:ETH+AMRx_FS2019+2019_T1/about
Roland Siegwart, Margarita Chli, Nick Lawrance

Roland Siegwart, Margarita Chli, Nick Lawrance 2019 | Introduction | Lecture Overview | 1
ASL
Autonomous mobile robot | your teachers
Roland Siegwart, ETH Zurich
Margarita Chli, ETH Zurich
Nick Lawrance ETH Zurich
Video segments
Marco Hutter, ETH Zurich
Davide Scaramuzza, Univ. of Zürich
Paul Furgale, Facebook Oculus Zurich
Martin Rufli, IBM Research

ASL
Autonomous mobile robot | about the course

 Running as an ETH-internal MOOC (Massive Open Online Course)

 Over 30 short video lectures that we call “segments”.
 The “segments” are complemented with:
 short questions for each segment to verify your understanding and progress
 various exercises (problem sets)
 videos showing the current state-of-the-art in the field
 Please register
 Textbook
On sale in LEE J206 for CHF 45
„Introduction to Autonomous Mobile Robots“
Roland Siegwart, Illah Nourbakhsh, Davide Scaramuzza, The MIT Press
 Other materials
 http://www.asl.ethz.ch/education/lectures/autonomous_mobile_robots/spring-2019.html

ASL
The Lecture
 We expect you to view and study the following elements beforehand:

 video segment
 relevant AMR book chapters
 problem sets and quizzes
 Lecture on Tuesday 10:15 – 12:00 in NO C 60
 We want your active participation!!
 Video Segments will not be repeated
 Focus on putting the learnt content into context
 Questions from students (in forum until Friday before the related lecture)
 go over difficult problems
 go a bit more in detail where needed (e.g. proofs of theorems, etc.)
 Exercises on Tuesday 14:15 – 16:00 in CAB G 11 (around every second week)
 Special exercises only supported for ETH students
ASL
Lecture Program

ASL
Exam
 Type
 Written session examination
 Language of examination
 English
 Course attendance confirmation required
 No
 Repetition
 The performance assessment is only offered in the session after the course unit.
Repetition only possible after re-enrolling.
 Mode of examination
 Multiple Choice and comprehension questions
 Calculations, similar to exercises, but simpler and solvable without computer
 Written aids
 4 A4-pages personal summary

ASL
Autonomous mobile robot | the key questions
 The three key questions in Mobile Robotics

 Where am I ?
?
 Where am I going ?
 How do I get there ?
 To answer these questions the robot has to

 have a model of the environment (given or autonomously built)
 perceive and analyze the environment
 find its position/situation within the environment
 plan and execute the movement

ASL
Autonomous mobile robot | the see-think-act cycle
knowledge, mission
data base commands
Localization “position“ Cognition
Map Building global map Path Planning
environment model
path
local map
Information Path
Motion Control
Extraction Execution
Perception
see-think-act actuator
raw data
commands
Sensing Acting
Real World
Environment
ASL
Motion Control | kinematics and motion control
 Wheel types and its constraints

 Rolling constraint
 no-sliding constraint (lateral)
 Motion control
⋯ , ,
⋮ , , ?
ASL
knowledge, mission
data base commands
environment model
path
local map
Information Path
Motion Control
Perception
raw data
commands
Sensing Acting
Real World
Environment
ASL
Perception | sensing Laser

GPS scanners
Omnidirectiona IMU
l
 Laser scanner camera
 time of flight Laser scanner
Security
Standard
switch
camera
 Camera
object Lens Wheel
Focal encoders
Plane
Focal Point
Focal Length: f
Roland Siegwart, Margarita Chli, Nick Lawrance https://www.youtube.com/watch?v=yvgPrZNp4So 2019 | Introduction | Lecture Overview | 11
ASL
Perception | information extraction
 Filtering / Edge Detection
 Keypoint Features
1 1 2
 features that are reasonably invariant to 1 6 3
rotation, scaling, viewpoint, illumination 5
4
1
 FAST, SURF, SIFT, BRISK, … 1
4 C 5
3
1 6
2
1 7
1 1 9 8
0
Image from [Rosten et al., PAMI 2010]
 Keypoint matching
 BRISK example
ASL
Virtual and Augmented Reality (VR / AR) on your Smartphone

ASL
Collaborative Visual-Inertial Navigation | teach and repeat
https://www.youtube.com/watch?v=pDlQXsOrgI4
https://www.youtube.com/watch?v=9PprNdIKRaw
ASL
From Perception to Understanding Localization “position“

global map
Cognition
environment model path

local map
Places / Situations Perception Real World

Environment
Motion Control
A specific room, a meeting situation, …

•Functional / Contextual
Fusing & Compressing Information
Servicing / Reasoning Relationships of Objects

• imposed
Objects • learned
• spatial / temporal/semantic
Doors, Humans, Coke bottle, car , …
Interaction •Models / Semantics

• imposed
• learned
Features
Lines, Contours, Colors, Phonemes, …
Navigation •Models
• imposed
Raw Data • learned
Autonomous Mobile Robots Vision, Laser, Sound, Smell, …
ASL
knowledge, mission
data base commands
environment model
path
local map
Information Path
Motion Control
Perception
raw data
commands
Sensing Acting
Real World
Environment
ASL
Localization | where am I?
 SEE: The robot queries its sensors

→ finds itself next to a pillar
 ACT: Robot moves one meter forward

 motion estimated by wheel encoders
 accumulation of uncertainty
 SEE: The robot queries its sensors

again → finds itself next to a pillar
 Belief update (information fusion)

ASL
knowledge, mission
data base commands
environment model
path
local map
Information Path
Motion Control
Perception
raw data
commands
Sensing Acting
Real World
Environment
ASL
Cognition | Where am I going ? How do I get there ?
Goal

ASL
Cognition | Where am I going ? How do I get there ?
 Global path planning

 Graph search
 Local path planning

 Local collision avoidance

ASL
knowledge, mission
data base commands
environment model
path
local map
Information Path
Motion Control
Perception
raw data
commands
Sensing Acting
Real World
Environment
ASL
Next generation of Robots

| mobile, smart, connected, adaptive and closer to humans
Industrial Robots Service Robots Cyborgs
https://www.youtube.com/watch?v=SeIoQy0oXjI

ASL
Robotics | challenges and technology drivers
 The challenges
 Seeing, feeling and
understanding the world
 Dealing with uncertain
and partially available
information
 Act appropriately onto
the environment
Willow Garage
 Technology drivers
| technology evolutions enable robotics revolutions
 Laser time-of-flight sensors
 Cameras and IMUs combined with required calculation power
 Torque controlled motors, “soft” actuation
 New materials
ASL
Autonomous Mobile Robots | Some recent examples

Examples – not part of MOOC Video Segment

ASL
Autonomous mobile robot | we invite you to join the course
https://www.youtube.com/watch?time_continue=4&v=M4CQ7VW7iZ0

ASL
Today | 3D laser sensors

Expensive, complex and cumbersome
 Google Self-Driving Car Project (status summer 2015)

 > 20 vehicles in use
 > 2,7 mio km, 1.5 mio km in autonomous mode
 > 11 accidents
 No people insured
https://www.youtube.com/watch?v=eJCR2TaeSFc
 Non of them caused by car control algorithm
ASL
Today | cameras (lane tracking, …)
https://www.youtube.com/watch?v=JmxDIuCIIcg https://www.youtube.com/watch?v=aGW4nRzx8Iw
 Detection and tracking of

 Lanes
 Street signs
 Other cars
 …
ASL
V-Charge | Automonous driving using close-to-market sensors

ASL
V-Charge | Autonomous driving using close-to-market sensors

ASL
Typical Situation
http://www.youtube.com/watch?v=wn2NfUH0G-Q
http://hamilton-baillie.co.uk
ASL
V-Charge Review 2 | Driving Demo

ASL
Autonomous Cars | roadmap

Fully autonomous
Car
(you can sleep)
Autonomous
car freeway
Driving Speed
Autonomous
car urban
Complexity of Environment → Perception and Interaction

ASL
V-Charge | the ultimate vision
 Mixed-traffic scenarios

ASL
NIFTi / TRADR – Urban Search and Rescuing
 Project goals
 Robotic help for Urban Search and Rescue
 UGV and UAV combined for scene exploration
 Yearly evaluation of system by firemen Omnicam
Rotating Laser
 Environment modeling
 Online 3D mapping from laser sensor
 Based on enhanced ICP released open-source
 Topological segmentation for human-robot interaction
https://www.youtube.com/watch?v=CMk4w4eRobg

ASL
Vision only UAV navigation www.sfly.ethz.ch/
 Swarm of small helicopters

 Vision only navigation (one camera, GPS denied)
 Fully autonomous with on-board computing
 Feature based visual SLAM
Proto 1
Proto 2
Proto 3
ASL
Navigation & Planning in Cluttered Environments
https://www.youtube.com/watch?v=rAJwD2kr7c0
ASL
Flourish – Aerial Data Collection and Analysis, and Automated Ground Intervention for Precision Farming
Spatio-Temporal Spectral Autonomous UAV landing Weed classification on UGV
Environment Modeling (Sunflower ~95% acc.)
https://youtu.be/5f1EtfW76Qc
Automated spraying and

stamping
UAV onboard weed detection

ASL
Collaborative Visual-Inertial Navigation

in collaboration with Prof. Marco Hutter
ASL
Beyond Mobility | PR2 robot from Willow Garage
Fold towels https://www.youtube.com/watch?v=gy5g33S0Gzo
Courtesy of
Clean-up
ASL
ETH Zurich | a melting pot for robotics, systems and control
Artificial Intelligence
Roger Gassert Simone Schürle

Joachim Buhmann Rehabilitation Engineering Responsive Biomedical
Machine Learning Lab Lab Systems Lab
Andreas Krause
Learning and Otmar Hilliges Walter Karlen
Adaptive Systems Advanced Interactive
Medical /Biomedical Robotics Mobile Health Systems
Technologies Lab Brad Nelson Lab
Multi-Scale Robotics
Lab
Autonomous Robots Robert Riener
Sensor Motor Systems
Roland Siegwart Marco Hutter Lab
Autonomous Systems Robotic Systems Emilio Frazzoli
Lab Information and Decision
Lab
Systems
Maryam
Margarita Chli
Vision for Robotics Kamgarpour
Melanie Zeilinger
Marc Pollefeys Lab
Intelligent Control
Automatic
Control Lab
Computer Vision Raffaello D’Andrea Systems
and Geometry Lab Dynamic Systems
Computer Vision and Control
Stelian Coros
Automation and Control
Computational John Lygeros
Luc van Gool Robotics Lab Automatic Control Lab
Computer Vision Roy Smith
Laboratory Automatic Control Lab
Florian Dörfler
Roland Siegwart, Margarita Chli, Nick Lawrance Automatic Control Lab | 40
ASL
Switzerland | a melting pot for robotics technology
Initiatives Spin-offs Some Industrial Collaborations
Center for
Learning Systems
ieLab

Roland Siegwart, Margarita Chli, Nick Lawrance Roland Siegwart | 41

0 ETH Lecture Introduction and Lecture Overview 2019

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

0 ETH Lecture Introduction and Lecture Overview 2019

Uploaded by

Copyright:

Available Formats

ASL

Autonomous Systems Lab

Mobile Robots | Introduction and Lecture Overview

Roland Siegwart, Margarita Chli, Nick Lawrance

Autonomous Mobile Robots

Autonomous mobile robot | your teachers

Roland Siegwart, ETH Zurich

Margarita Chli, ETH Zurich

Nick Lawrance ETH Zurich

Marco Hutter, ETH Zurich

Davide Scaramuzza, Univ. of Zürich

Paul Furgale, Facebook Oculus Zurich

Martin Rufli, IBM Research

Autonomous mobile robot | about the course

 Running as an ETH-internal MOOC (Massive Open Online Course)

Autonomous Mobile Robots

 We expect you to view and study the following elements beforehand:

Autonomous Mobile Robots

Autonomous Mobile Robots

Autonomous mobile robot | the key questions

 The three key questions in Mobile Robotics

 To answer these questions the robot has to

Autonomous Mobile Robots

Autonomous mobile robot | the see-think-act cycle

Motion Control | kinematics and motion control

 Wheel types and its constraints

Autonomous mobile robot | the see-think-act cycle

Perception | sensing Laser

Perception | information extraction

 Filtering / Edge Detection

Virtual and Augmented Reality (VR / AR) on your Smartphone

Autonomous Mobile Robots

Collaborative Visual-Inertial Navigation | teach and repeat

From Perception to Understanding Localization “position“

environment model path

Places / Situations Perception Real World

A specific room, a meeting situation, …

Servicing / Reasoning Relationships of Objects

Interaction •Models / Semantics

Autonomous mobile robot | the see-think-act cycle

 SEE: The robot queries its sensors

 ACT: Robot moves one meter forward

 SEE: The robot queries its sensors

 Belief update (information fusion)

Autonomous mobile robot | the see-think-act cycle

Cognition | Where am I going ? How do I get there ?

Autonomous Mobile Robots

Cognition | Where am I going ? How do I get there ?

 Global path planning

 Local path planning

Autonomous Mobile Robots

Autonomous mobile robot | the see-think-act cycle

Next generation of Robots

Industrial Robots Service Robots Cyborgs

Autonomous Mobile Robots

Robotics | challenges and technology drivers

Autonomous Mobile Robots | Some recent examples

Autonomous Mobile Robots

Autonomous mobile robot | we invite you to join the course

Autonomous Mobile Robots

Today | 3D laser sensors

 Google Self-Driving Car Project (status summer 2015)

Today | cameras (lane tracking, …)