Multi-Agent Path Finding for Robotics: Progress and

Challenges
Workshop on Multi-Agent Path Finding — IJCAI-20

Wolfgang Hönig

January 7, 2021
Motivation

Source: EuroGamer Source: Robotics Business Review

Computer Games Robotics / Autonomy 1

Overview

1. Multi-Agent Path Finding Overview

2. MAPF for Robotics

3. Conclusion

2
Multi-Agent Path Finding Overview
Multi-Agent Path Finding (MAPF)

• Given: map of environment, start and goal position for each agent
• Goal: collision-free schedule of all agents,
• Optimality: minimize sum of actions or makespan (time until last agent reaches
goal)

3
Naı̈ve Approach 1: Prioritized/Sequential Planning

• In general incomplete. Counter-example:

1 2

• Environments with well-formed infrastructure property (Čáp et al., 2015): complete

but not optimal

4
Naı̈ve Approach 2: Joint-Space Planning

• Search-space grows exponentially with the number of agents

• Does not scale to long time horizons or high number of agents

5
Current Approaches

Reduction-based Search-based Rule-based

• Integer Linear • M ∗ (Wagner et al., 2011) • Push-and-Rotate
Program (ILP) (Yu • Conflict-Based Search (Wilde et al., 2014)
et al., 2016) (CBS) (Sharon et al.,
• Boolean satisfiability 2015)
(SAT) (Surynek, 2012)

Complete, Optimal or
Complete, Optimal w -bounded suboptimal Complete, suboptimal

6
Variants & Interesting Theoretical Results

Labeled Unlabeled K-Colored

• Minimize sum-of-cost: • Minimize makespan:

NP-Hard (Surynek, Polynomial (Yu et al.,
2010) 2016)

• Minimize makespan: • Minimize makespan:

NP-Hard, even to NP-hard, even to
approximate with approximate with
factor less than 4/3 factor less than 4/3
(Ma et al., 2016)
(Ma et al., 2016)

7
MAPF for Robotics
Agents ⇒ Robots

Source: New Venturist Source: Örebro University

Challenges:

1. Movement capabilities (path ⇒ motion)

2. Dynamic changes (environment / tasks)
3. Partially unknown environment (sensing / observability)
4. Inter-robot interactions (e.g., aerial robots)

Multi-Agent Path Finding ⇒ Multi-Robot Motion Planning 8

Background: Multi-Robot Motion Planning

Meta-Robot Decoupling
(∼ AI: Reduction-based) (∼ AI: Search-based)

• Mixed Integer Quadratic • Priority-based planning (Buckley, 1989;

Programming (MIQP) (Mellinger et al., J. P. v. d. Berg et al., 2005; Bennewitz
2012) et al., 2001)

• Velocity profiles (Peng et al., 2005)

Collision Avoidance Sampling

(∼ AI: Rule-based)

• ORCA (J. v. d. Berg et al., 2009) • dRRT* (Shome et al., 2020)

9
Challenge 1: Movement Capabilities

Robots are subject to Kinodynamic constraints:

Kinematic contraints Geometric constraints (e.g. robot shape, obstacles)

Dynamic constraints Temporal constraints (e.g. velocity, acceleration)

Both might be coupled (e.g. car turning)

10
MAPF-POST (Hönig et al., 2016): Robot Model

• Robot can follow timed waypoints

• Constant speed between waypoints
• Speed can only change at waypoints
• No acceleration constraints

11
MAPF-POST (Hönig et al., 2016): Temporal Plan Graph

s1 s2 g2 g1
1 2
A B C D E

r1 A B C D E

B C F C D
r2

• Type 1 edges: robots follow discrete path in sequence (black)

• Type 2 edges: dependencies between robots (color)
• Construction in polynomial time
12
MAPF-POST (Hönig et al., 2016): Augmented Temporal Plan Graph

• Simple TPG does not enforce safety distance

• Original TPG:
A B C D E

B C F C D

• Add additional “safety markers” in augmented TPG (δ = 0.75)

A B C D E

B C F C D

13
MAPF-POST (Hönig et al., 2016): Simple Temporal Network

• Encode non-uniform edge length and velocity bounds

• Edge annotation: [minimum time to traverse, maximum time to traverse]

[1, ∞] [1, ∞] [1, 8] [1, ∞]

[3, ∞] [3, ∞] [3, 24] [3, ∞]
0] A B C D E [0, ∞
[0, ]

[0, ∞
]

[0, ∞
]

[0, ∞
]

[0, ∞
[0, ∞

[0, ∞

[0, ∞
[0, ∞
XS XF

]
]
]

]
[0, ]
0] ∞
B C F C D [0,
[12, ∞] [12, ∞] [12, ∞] [12, 24]
[4, ∞] [4, ∞] [4, ∞] [4, 8]

Edge lengths: 1 m; δ = 0.75 m; minimum speed for edge (C , D): 1/32 m/s
maximum speed robot 1: 1/4 m/s; maximum speed robot 2: 1/16 m/s

14
MAPF-POST (Hönig et al., 2016): Properties

1. User-specified guaranteed safety distance δ, with respect to the graph distance

2. Guaranteed solution if lower speed limit is zero
3. Polynomial runtime

15
MAPF/C (Hönig et al., 2018): Overview

Discrete planning

Input Output
1. Roadmap 2. Conflict 3. MAPF/C
Generation Annotation Solver

Continuous post-processing

4. Spatial 5. Trajectory
Partition Optimization
16
MAPF/C (Hönig et al., 2018): Step 1: Roadmap Generation

• Desired properties:
1. Connectivity matches connectivity of free
space
2. Shortest paths approximate continuous
shortest paths
3. Sparse, for computational efficiency
• We use the SPARS algorithm (Dobson et al.,
2014)
• Generate a roadmap (similar to PRM)
• Only keep “important” edges and vertices

17
MAPF/C (Hönig et al., 2018): Step 2: Conflict Annotation

• Generalized roadmap and physical robot model creates new constraints

• Vertex-Edge

• Vertex-Vertex
• Edge-Edge
• Swept ellipsoid collision checking using a quadratic program
• Computation can be done as pre-processing, given the roadmap and robot model
18
MAPF/C (Hönig et al., 2018): Step 3: Multi-Agent Path Finding with Generalized
Conflicts (MAPF/C)

• MAPF/C is MAPF extended by the generalized vertex-vertex, edge-edge, and

vertex-edge conflicts
• Input:
• Graph of the environment (roadmap)
• Start and (potential) goal vertices
• Conflict sets: conVV (v ), conEE (e), conEV (e)
• Output: schedule for each robot, such that all constraints are fulfilled
• NP-hard, even in the unlabeled case
• We developed several solvers:
• Extension of Enhanced Conflict-Based Search (ECBS)
• Integer Linear Program (ILP)

19
Video

https://youtu.be/7KIa9FlmbRc
20
LA-MAPF (Li et al., 2019): MAPF for Large Agents

• Generalized conflicts increase chance of collision

• MC-CBS: add multiple constraints per CT-node in CBS

• Reduces runtime by 2 – 3 orders of magnitude

21
Other Approaches: State Lattices

• Successfully used for single-robot motion planning (Pivtoraiko, 2012; Liu et al., 2017)
• Difficult to select/compute motion primitives
22
Challenge 2: Dynamic Changes

Robots are subject to unforeseeable changes:

Task Changes New tasks (e.g., warehouse automation)
Environment Changes Blocked passages (e.g., maintenance work, robot failures)
Optimality
In these cases, it is difficult to define what optimal behavior is, since dynamic
changes are apriori unknown. 23
MAPD (Ma et al., 2017): Multi-Agent Pickup and Delivery

• Task: pick up item at pickup location and

deliver to delivery location
• (finite) task set can be dynamically updated
(task stream)
• Service time of a task: tfinished − tadded
• Completeness: service time of all tasks finite
• Optimality: minimum average service time
Not every MAPD instance is solvable (like MAPF)

24
MAPD Algorithms

• Token Passing (TP): sequential planning in a decentralized fashion (Ma et al., 2017)
• Token Passing with Task Swaps (TPTS): allow to swap “unexecuted” tasks
between agents (Ma et al., 2017)
• TP-SIPPwRT: Combination of TP, Safe Interval Path Planning (SIPP), and
MAPF-POST (Ma et al., 2019)
• Rolling-Horizon Collision Resolution (RHCR) (Li et al., 2020):
1. Allow agents to be assigned to a sequence of goal locations
2. Resolve conflicts only for a fixed time horizon

25
MAPF-ADG (Hönig et al., 2019)

Similar idea as in MAPF-POST

1. Use any MAPF solver A 3 2 B
2. Post-process output into an action dependency graph
• Vertices: refer to actions (e.g., move, turn, yield)
C 1 D
• Edges: temporal dependencies between actions
E F
• Polynomial construction time

r1 (0, Y, D, D) (1, ↑, D, C ) (2, , C , C )

r2 (0, , B, B) (1, ↑, B, D) (2, Y, D, D) (3, ↑, D, F )

r3 (1, ↑, A, B) (2, , B, B) (3, ↑, B, D)

26
MAPF-ADG (Hönig et al., 2019): Online Update

• Robot model: Each robot has a queue of actions, controllers to

execute actions, and sensing to detect unforeseen obstacles A 3 B
2
• Keep track of completion status of each vertex/action (finished, C 1 D
enqueued, staged)
• Only enqueue (i.e., send to robot) if dependent actions are
E F
finished

r1 (0, Y, D, D) (1, ↑, D, C ) (2, , C , C )

r2 (0, , B, B) (1, ↑, B, D) (2, Y, D, D) (3, ↑, D, F )

r3 (1, ↑, A, B) (2, , B, B) (3, ↑, B, D)

27
MAPF-ADG (Hönig et al., 2019): Lifelong Operation

• Overlap execution and (incremental) re-planning

• Trigger planning if at least one robot will reach its goal soon
• Find commit cut (defines actions that can be safely executed by the robots)
• Update MAPF schedule by only changing vertices after commit cut

cc
1
r
cc
2
r
cc
3
r

28
kR MAPF (Atzmon et al., 2020)

• Delay: agent cannot execute desired move action and performs a wait action
instead

• k-Robust MAPF: produce a schedule that is valid for up to k delays per agent
• Can be solved in A*, CBS, and SAT frameworks

29
Challenge 3: Partially Unknown Environment

Robots have a limited observability:

Environment Sensing robots might
sense environment
inaccurately
Communication Large robot teams
might not be able to
communicate fully
Relationship to Challenge 2 (Dynamic Changes)
Some solution approaches presented for challenge 2 also address this challenge
partially.

30
Decentralized Hybrid Motion Planning (Şenbaşlar et al., 2018)

Continuous
Discrete planning
post-processing

Execute for δt and start next iteration

• Reactive, fully distributed

• Not complete, not optimal
• (Some) safety guarantees
31
GLAS (Rivière et al., 2020): Motivation

Centralized Planners Decentralized Planners

• Complete • Incomplete
• Full Information • Local Information 32
GLAS (Rivière et al., 2020): Overview

Globally Planned Examples Local Observations at t = 4 s

Imitation Learning
Policy

Deploy

33
GLAS (Rivière et al., 2020): Dataset Generation i

34
GLAS (Rivière et al., 2020): Dataset Generation ii

35
GLAS (Rivière et al., 2020): Model Architecture

Input Differentiable Safety Module

Deep Set (Obstacles)

Deep Set (Neighbors)

36
Video

https://youtu.be/RZmF-1q1fVg
37
Challenge 4: Inter-robot Interactions

Robots might interact with each other:

Multi-Robot Tasks e.g., collaborative transport
Single-Robot Tasks e.g., downwash for aerial vehicles

Source: ETH Zürich

38
Neural-Swarm2 (Shi et al., 2020): Interaction Forces i

• Interaction force magnitude is significant

• What happens for more than 2 multirotors?
• How can we model the interaction force?

Data collection + Deep Learning to approximate fa (N (i) )

39
Neural-Swarm2 (Shi et al., 2020): Interaction Forces ii

40
Neural-Swarm2 (Shi et al., 2020): Motion Planning i

• Sequential planning with simplified double integrator dynamics with interaction

prediction
• Stage 1: Sampling-based (extension of AO-RRT (Hauser et al., 2016))
• Stage 2: Optimization-based (extension of SCP)

41
Neural-Swarm2 (Shi et al., 2020): Motion Planning ii
10
Initial trajectory of neighbor Rejected motions
Search tree Solution

| fa | [g]
Z

0
42
Y
Conclusion
Conclusion

• Challenges when applying MAPF to 1. How to close/reduce the

robots: discrete/continuous gap?
1. Motion limitations
2. How to handle teams with arbitrary
2. Dynamic task changes
3. Observability dynamics ẋ i = f (x i , u i , N i )
4. Inter-robot Interactions efficiently?
• Progress has been made towards all 3. How to consider uncertainty (tasks,
challenges observation, actions) holistically?

43
Acknowledgments

Nora Ayanian Sven Koenig T. K. Satish Kumar Gaurav S. Sukhatme Soon-Jo Chung

(and many others, see reference list)

44
 Multi-Agent Path Finding for Robotics: Progress and
Challenges

• More Information: • Contact:

http://mapf.info whoenig@caltech.edu
44
References i

Atzmon, Dor, Roni Stern, Ariel Felner, Glenn Wagner, Roman Barták, and Neng-Fa Zhou (2020). “Robust
Multi-Agent Path Finding and Executing”. In: J. Artif. Intell. Res. 67, pp. 549–579. doi:
10.1613/jair.1.11734 (cit. on p. 32).
Bennewitz, Maren, Wolfram Burgard, and Sebastian Thrun (2001). “Optimizing Schedules for Prioritized Path
Planning of Multi-Robot Systems”. In: IEEE International Conference on Robotics and Automation (ICRA).
IEEE, pp. 271–276. doi: 10.1109/ROBOT.2001.932565 (cit. on p. 12).
Berg, Jur P. van den and Mark H. Overmars (2005). “Prioritized motion planning for multiple robots”. In:
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, pp. 430–435. doi:
10.1109/IROS.2005.1545306 (cit. on p. 12).
Berg, Jur van den, Stephen J. Guy, Ming C. Lin, and Dinesh Manocha (2009). “Reciprocal n-Body Collision
Avoidance”. In: International Symposium on Robotics Research (ISRR). Vol. 70. Springer Tracts in Advanced
Robotics. Springer, pp. 3–19. doi: 10.1007/978-3-642-19457-3_1 (cit. on p. 12).
Buckley, Stephen J. (1989). “Fast motion planning for multiple moving robots”. In: IEEE International
Conference on Robotics and Automation (ICRA). IEEE, pp. 322–326. doi: 10.1109/ROBOT.1989.100008
(cit. on p. 12).
References ii
Čáp, Michal, Peter Novák, Alexander Kleiner, and Martin Selecký (2015). “Prioritized Planning Algorithms for
Trajectory Coordination of Multiple Mobile Robots”. In: IEEE Transactions on Automation Science and
Engineering 12.3, pp. 835–849. doi: 10.1109/TASE.2015.2445780 (cit. on p. 6).
Dobson, Andrew and Kostas E. Bekris (2014). “Sparse roadmap spanners for asymptotically near-optimal
motion planning”. In: International Journal of Robotics Research (IJRR) 33.1, pp. 18–47. doi:
10.1177/0278364913498292 (cit. on p. 20).
Hauser, Kris and Yilun Zhou (2016). “Asymptotically Optimal Planning by Feasible Kinodynamic Planning in a
State-Cost Space”. In: IEEE Trans. Robotics 32.6, pp. 1431–1443. doi: 10.1109/TRO.2016.2602363 (cit. on
p. 44).
Hönig, Wolfgang, Scott Kiesel, Andrew Tinka, Joseph W. Durham, and Nora Ayanian (2019). “Persistent and
Robust Execution of MAPF Schedules in Warehouses”. In: IEEE Robotics and Automation Letters (RA-L)
4.2, pp. 1125–1131. doi: 10.1109/LRA.2019.2894217 (cit. on pp. 29–31).
Hönig, Wolfgang, T. K. Satish Kumar, Liron Cohen, Hang Ma, Hong Xu, Nora Ayanian, and Sven Koenig
(2016). “Multi-Agent Path Finding with Kinematic Constraints”. In: International Conference on Automated
Planning and Scheduling (ICAPS). Awarded Best Paper in robotics track. AAAI Press, pp. 477–485. url:
http://www.aaai.org/ocs/index.php/ICAPS/ICAPS16/paper/view/13183 (cit. on pp. 14–18).
References iii
Hönig, Wolfgang, James A. Preiss, T. K. Satish Kumar, Gaurav S. Sukhatme, and Nora Ayanian (2018).
“Trajectory Planning for Quadrotor Swarms”. In: IEEE Transactions on Robotics, Special Issue on Aerial
Swarm Robotics 34.4, pp. 856–869. doi: 10.1109/TRO.2018.2853613 (cit. on pp. 19–22).
Li, Jiaoyang, Pavel Surynek, Ariel Felner, Hang Ma, T. K. Satish Kumar, and Sven Koenig (2019).
“Multi-Agent Path Finding for Large Agents”. In: AAAI Conference on Artificial Intelligence, pp. 7627–7634.
doi: 10.1609/aaai.v33i01.33017627 (cit. on p. 24).
Li, Jiaoyang, Andrew Tinka, Scott Kiesel, Joseph W. Durham, T. K. Satish Kumar, and Sven Koenig (2020).
“Lifelong Multi-Agent Path Finding in Large-Scale Warehouses”. In: International Conference on
Autonomous Agents and Multiagent Systems (AAMAS), pp. 1898–1900. url:
https://dl.acm.org/doi/abs/10.5555/3398761.3399020 (cit. on p. 28).
Liu, Sikang, Nikolay Atanasov, Kartik Mohta, and R. Vijay Kumar (2017). “Search-based motion planning for
quadrotors using linear quadratic minimum time control”. In: IEEE/RSJ International Conference on
Intelligent Robots and Systems (IROS), pp. 2872–2879. doi: 10.1109/IROS.2017.8206119 (cit. on p. 25).
Ma, Hang, Wolfgang Hönig, T. K. Satish Kumar, Nora Ayanian, and Sven Koenig (2019). “Lifelong Path
Planning with Kinematic Constraints for Multi-Agent Pickup and Delivery”. In: AAAI Conference on
Artificial Intelligence, pp. 7651–7658. doi: 10.1609/aaai.v33i01.33017651 (cit. on p. 28).
References iv
Ma, Hang, Jiaoyang Li, T. K. Satish Kumar, and Sven Koenig (2017). “Lifelong Multi-Agent Path Finding for
Online Pickup and Delivery Tasks”. In: Conference on Autonomous Agents and MultiAgent Systems
(AAMAS). ACM, pp. 837–845. url: http://dl.acm.org/citation.cfm?id=3091243 (cit. on pp. 27, 28).
Ma, Hang, Craig A. Tovey, Guni Sharon, T. K. Satish Kumar, and Sven Koenig (2016). “Multi-Agent Path
Finding with Payload Transfers and the Package-Exchange Robot-Routing Problem”. In: AAAI Conference
on Artificial Intelligence. AAAI Press, pp. 3166–3173. url:
http://www.aaai.org/ocs/index.php/AAAI/AAAI16/paper/view/12437 (cit. on p. 9).
Mellinger, Daniel, Aleksandr Kushleyev, and Vijay Kumar (2012). “Mixed-integer quadratic program trajectory
generation for heterogeneous quadrotor teams”. In: IEEE International Conference on Robotics and
Automation (ICRA). IEEE, pp. 477–483. doi: 10.1109/ICRA.2012.6225009 (cit. on p. 12).
Peng, Jufeng and Srinivas Akella (2005). “Coordinating Multiple Robots with Kinodynamic Constraints Along
Specified Paths”. In: International Journal of Robotics Research (IJRR) 24.4, pp. 295–310. doi:
10.1177/0278364905051974 (cit. on p. 12).
Pivtoraiko, Mikhail (Feb. 2012). “Differentially Constrained Motion Planning with State Lattice Motion
Primitives”. PhD thesis. Pittsburgh, PA: Carnegie Mellon University. url:
http://ri.cmu.edu/pub_files/2012/2/pivtoraiko_thesis.pdf (cit. on p. 25).
References v
Rivière, Benjamin, Wolfgang Hönig, Yisong Yue, and Soon-Jo Chung (2020). “GLAS: Global-to-Local Safe
Autonomy Synthesis for Multi-Robot Motion Planning With End-to-End Learning”. In: IEEE Robotics
Autom. Lett. 5.3, pp. 4249–4256. doi: 10.1109/LRA.2020.2994035 (cit. on pp. 35–39).
Şenbaşlar, Baskın, Wolfgang Hönig, and Nora Ayanian (2018). “Robust Trajectory Execution for Multi-Robot
Teams Using Distributed Real-time Replanning”. In: Int. Symp. on Distributed Autonomous Robotic Systems
(DARS). Accepted. To Appear. url: http://act.usc.edu/publications/Senbaslar_DARS2018.pdf
(cit. on p. 34).
Sharon, Guni, Roni Stern, Ariel Felner, and Nathan R. Sturtevant (2015). “Conflict-based search for optimal
multi-agent pathfinding”. In: Artificial Intelligence 219, pp. 40–66. doi: 10.1016/j.artint.2014.11.006
(cit. on p. 8).
Shi, Guanya, Wolfgang Hönig, Xichen Shi, Yisong Yue, and Soon-Jo Chung (2020). “Neural-Swarm2: Planning
and Control of Heterogeneous Multirotor Swarms using Learned Interactions”. In: CoRR abs/2012.05457.
arXiv: 2012.05457. url: https://arxiv.org/abs/2012.05457 (cit. on pp. 42–45).
Shome, Rahul, Kiril Solovey, Andrew Dobson, Dan Halperin, and Kostas E. Bekris (2020). “dRRT* : Scalable
and informed asymptotically-optimal multi-robot motion planning”. In: Auton. Robots 44.3-4, pp. 443–467.
doi: 10.1007/s10514-019-09832-9 (cit. on p. 12).
References vi
Surynek, Pavel (2010). “An Optimization Variant of Multi-Robot Path Planning Is Intractable”. In: AAAI
Conference on Artificial Intelligence. AAAI Press. url:
http://www.aaai.org/ocs/index.php/AAAI/AAAI10/paper/view/1768 (cit. on p. 9).
— (2012). “Towards Optimal Cooperative Path Planning in Hard Setups through Satisfiability Solving”. In:
Pacific Rim International Conference on Artificial Intelligence (PRICAI). Vol. 7458. Lecture Notes in
Computer Science. Springer, pp. 564–576. doi: 10.1007/978-3-642-32695-0_50 (cit. on p. 8).
Wagner, Glenn and Howie Choset (2011). “M*: A complete multirobot path planning algorithm with
performance bounds”. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
IEEE, pp. 3260–3267. doi: 10.1109/IROS.2011.6095022 (cit. on p. 8).
Wilde, Boris de, Adriaan ter Mors, and Cees Witteveen (2014). “Push and Rotate: a Complete Multi-agent
Pathfinding Algorithm”. In: Journal of Artificial Intelligence Research (JAIR) 51, pp. 443–492. doi:
10.1613/jair.4447 (cit. on p. 8).
Yu, Jingjin and Steven M. LaValle (2016). “Optimal Multirobot Path Planning on Graphs: Complete Algorithms
and Effective Heuristics”. In: IEEE Transactions on Robotics 32.5, pp. 1163–1177. doi:
10.1109/TRO.2016.2593448 (cit. on pp. 8, 9).