2022 Latin American Robotics Symposium (LARS), 2022 Brazilian Symposium on Robotics (SBR), and 2022 Workshop on Robotics
in Education (WRE) | 978-1-6654-6280-8/22/$31.00 ©2022 IEEE | DOI: 10.1109/LARS/SBR/WRE56824.2022.9995874
Autonomous Robot Navigation in Crowd
1st Paulo de Almeida Afonso
Programa de Pós-Graduação em Computação
Centro de Desenvolvimento Tecnológico (CDTec)
Universidade Federal de Pelotas (UFPEL)
Pelotas, Brasil
paafonso@inf.ufpel.edu.br
Abstract—This study presents a review of the literature that
addresses the problem of autonomous navigation in crowded
indoor environments. Navigation in this type of environment is
a particularly challenging task because a robot must be able
to navigate autonomously without endangering nearby people.
We analyze a few selected studies in this field published in
the last seven years. The analysis shows that a combination of
different techniques is necessary for safe navigation in this type
of environment. Therefore we follow a line of previous studies to
seek new solutions, or investigate and improve existing solutions
to address the aforementioned problem.
Index Terms—autonomous navigation, indoor environments,
crowded environment, crowd navigation
I. INTRODUCTION
Studies related to the integration of robots in daily human
activities in places, such as airports, museums, and malls, have
shown that autonomous navigation in this type of environment
is a challenging task [1]–[4].
In crowded scenarios, movement of people can generate
obstructions, sensing hindrances, and perception impairment
of the robot in relation to its position in the environment
[5]. Moreover, the uncertainty of human behavior can lead to
unsafe situations for the robots and people during navigation.
Faced with this challenge, several solutions have been ex-
plored for robots to navigate efficiently and safely, considering
the people movements around them.
This study presents a literature review on how autonomous
navigation in crowded internal scenarios is addressed. The
articles published in the last seven years in important scientific
journals and conferences in the field of robotics were consid-
ered in this study.
The analysis of the selected articles shows that the existing
approaches are mainly divided into two methods: reactive
methods, in which the decision-making process of the agent
is initiated by identifying an imminent collision [6], [7]; and
predictive methods, in which the collision-free trajectories or
future actions of pedestrians are predicted through behavioral
modeling of humans [8]–[10].
Additionally, there are techniques that consider the solutions
based on learning [11]–[13], which sometimes combine with
978-1-6654-6280-8/22/$31.00 ©2022 IEEE
139
Authorized licensed use limited to: FUDAN UNIVERSITY. Downloaded on February 25,2023 at 04:33:29 UTC from IEEE Xplore. Restrictions apply.
2nd Paulo Roberto Ferreira Jr.
Programa de Pós-Graduação em Computação
Centro de Desenvolvimento Tecnológico (CDTec)
Universidade Federal de Pelotas (UFPEL)
Pelotas, Brasil
paulo@inf.ufpel.edu.br
reactive algorithms [14] or use strategies aimed at social
navigation [15]–[18].
The remainder of this paper is organized as follows. Section
II presents the research methodology adopted in this study.
Section III presents the main reactive methods that include
methods based on reinforcement learning (RL) and deep rein-
forcement learning (DRL). Section IV presents the predictive
methods, which includes, the most commonly used algorithms
and some of their derivations that represent efficient solution
modeling of human beings and prediction of trajectories.
The obtained results are discussed in Section V. Finally, the
conclusions are presented, and the references are listed.
II. METHODOLOGY
The main sources used in this study were the CAPES
Journal Portal, digital library IEEEXplore, Google Scholar,
and Scopus Database. By searching using relevant terms and
keywords (autonomous navigation, crowded scenarios, crowd
navigation, collision avoidance in crowd, robot navigation in
crowded), papers published in the last seven years in important
scientific journals and conferences in the field of robotics
were selected, such as: International Journal of Robotics
Research, Autonomous Robots, International Conference on
Autonomous Robot Systems and Competitions (ICARSC),
International Conference on Robotics and Automation (ICRA)
and International Conference on Intelligent Robots and Sys-
tems (IROS).
The selected papers were then classified based on their tech-
niques. Those directly related to the object of this study were
first ranked. Their references were consulted, and according
to the proposed approach, the papers were analyzed and the
observations were noted.
III. R EACTIVE M ETHODS
A. Reinforcement Learning Based
Traditional Reinforcement Learning (RL) methods can be
used to model reactive behaviors by processing the sensor
reading data and the relative positions of the objects, which
are key elements of a navigation environment. In this method,
https://www.periodicos.capes.gov.br/
https/iexpee.ie.ieorg/
https://scholar.google.com.br/
https//www.scopus.com/
collision avoidance is performed based on the speed and
direction of a model; the navigation task is performed by
processing the received data, issuing linear and angular speed
commands to control the robot, and to avoiding obstacles [19].
Generally, training data are preprocessed through a neural
network, and further, the resource map extracted from the
supervised learning model is used as data input to the network,
resulting in a set of commands to determine the actions of the
robot [20].
The RL problems involve learning the task and mapping
situations to actions with the aim of maximizing a reward value
[21], [22]. To use RL in situations addressing real-world com-
plexity, agents should derive efficient representations of the
environment and use the received sensory inputs to generalize
past experiences for application in future situations [23].
The Q-learning algorithm proposed by Watkins and Dayan
[24] is one of the most popular RL algorithms. Its application
has been adopted in several studies for planning different
navigation and collision avoidance models [25], [26].
There are four basic components of RL: agent, environment,
reward, and action. A typical RL algorithm operates with a
limited knowledge of the environment and feedback on the
quality of decisions [27].
Fig. 1. Reinforcement Learning
At each step of the interaction with the T environment, the
B agent receives an input i as an indication of the current
state s, and chooses an a action to generate an output. The
action changes the state of the environment, and the value of
this state transition is communicated to the agent through a r
reinforcing signal [28, Fig. 1].
After multiple repetitions of this process, the agent is able
to choose actions that tend to increase the sum of the values
of the reinforcement signal. An agent can learn to do this
over time through systematic trial and error guided by a wide
variety of algorithms.
B. Deep Reinforcement Learning Based
The DRL is a combination of RL and deep learning (DL).
Its use has been researched for a variety of applications, such
as games [29], autonomous navigation in indoor environments
[30], [31], driving autonomous vehicles in urban scenarios
[32], [33], recognition of static and mobile objects [34], and
collision avoidance during autonomous navigation [35]–[38].
140
Authorized licensed use limited to: FUDAN UNIVERSITY. Downloaded on February 25,2023 at 04:33:29 UTC from IEEE Xplore. Restrictions apply.
Unlike purely reactive methods, in a DRL approach we seek
to code cooperative behaviors by learning a value function
[39], [40], or learning from the experiences of various agents
during the training steps [41]. These strategies allow the
algorithms to choose the actions to be performed, based on the
observations of an arbitrary number of nearby agents, without
assuming that other agents follow any specific behavioral
model.
In DRL approaches, different algorithms were combined
to implement collision prevention policies. Studies based on
decentralized environments demonstrate that the data obtained
from each agent belonging to the environment can be applied
to deep neural networks for the planning stage, whereas RL
algorithms, such as proximal policy optimization (PPO) [42]–
[44], can be used in the training stage to update the collision
avoidance policy.
To improve the algorithm performance and reduce the
computational cost, artificially generated maps and training
datasets containing real-world samples can be used [45], or
independent decision models can be created using repeated
offline simulations [46].
It is possible to store experience transactions using a mem-
ory module to minimize the learning time [47]. The compu-
tation time can also be reduced by eliminating unnecessary
repetitions during the network training step.
C. Artificial Potential Field Based
Solutions based on artificial potential field algorithms that
combine different techniques modifications with the traditional
algorithm have been proposed to improve the planning of paths
and to avoid obstacles [48].
Modifications to the Artificial Potential Field (APF) can
be adopted to create new attractive force-points and help the
moving robot to escape from the local minimum [49, Fig. 2].
Fig. 2. T-APF
In this type of solution, the robot makes the route using
the traditional APF algorithm, and upon reaching a local
minimum, a function based on the modified APF is applied to
stimulate the robot to change its trajectory and then resume
the route towards its goal.
Sampling-based techniques to identify collision-free paths
in a dynamic environments can be combined with artificial
power-field planning methods for navigation in the dynamic
environments [50], [51]. The solution consists of calculating
a path without static obstacle collisions that can be used as
an intermediate attribute to achieve this goal. To improve the
security of the algorithm, a repulsive potential field can be
incorporated for each moving obstacle based on precomputed
stochastic sets.
A modification of the repulsion field function by introducing
a relative distance value between the target point and robot
was presented by [52]. The solution adopted a combination
of APF with the fuzzy control algorithm, creating a function
to increase the perception of the mobile robot beyond the
distance from the obstacles. As the robot approaches the goal,
the repulsion force is updated by a regulating factor tending
to zero, until it reaches the goal.
D. Velocity Obstacle Based
The concept of a velocity obstacle based on the geometric
structure of the collision cone was first presented by [53].
In the proposed structure, obstacles are observed in the local
horizontal plane (XY) of the agent with its flat cross-section
centered in p⃗j , as shown in Fig. 3 [54]:
Fig. 3. Velocity Obstacles
A study by [54] analyzed and compared the performance
of several well-established approaches for avoiding collisions
in non-cooperative multiagent systems, the algorithms, Ve-
locity Obstacle - VO [53], Reciprocal Velocity Obstacle -
RVO [55], Hybrid Reciprocal Velocity Obstacle - HRVO [56]
and Optimal Reciprocal Collision Avoidance - ORCA [57],
were studied in several scenarios with different levels of
difficulty. Analysis of his results showed that reactive collision
avoidance methods may be sufficient to avoid collisions in
environments with no communication between agents. The
HRVO and ORCA methods were proved to be more efficient
in dense environments when handling trajectory uncertainties.
The ORCA method also exhibiteds smoother trajectories and
better computing times.
In [58] was presented an extension of the VO method,
aiming to determine the safest path between the robot’s current
and destination positions. In the proposed solution, the authors
assumed that the robot speeds and obstacles are known or mea-
surable. The SVO method uses a velocity-vector component to
calculate the safest speed at each sampling step. Depending on
141
Authorized licensed use limited to: FUDAN UNIVERSITY. Downloaded on February 25,2023 at 04:33:29 UTC from IEEE Xplore. Restrictions apply.
the need for the application, and time constraints, the method
can be adopted to ensure greater safety during navigation.
IV. P REDICTIVE M ETHODS
A. Based on Human Behavior
Some studies have highlighted that purely reactive obstacle
prevention techniques may not be sufficient for solving naviga-
tion problems in dynamic environments [59], [60]. According
to the authors, in this type of scenario, the robot should
coexist, cooperate with humans, and focus on other obstacles
in motion.
Modern structures of DRL and human-robot interaction
(HRI) can be used to encode the previous knowledge of hu-
mans, which is achieved by introducing cooperative behavioral
characteristics to the actions of the robot and reinforcing safety
during navigation [61]. In this type of solution, the robot learns
the appropriate social navigation using data obtained from
sensor inputs, whereas the reward update can occur based on
human feedback or actions performed during interaction with
the environment.
Fig. 4. RL Architecture for social navigation and HRI.
Figure 4 represents a learning architecture proposed by [61].
The data obtained are processed by a neural network and
used for path prediction and calculating proxemic distances for
social navigation and collision avoidance, a solution based on a
called Composite Reinforcement Learning (CRL) framework.
Solutions based on pedestrian behavior can also be imple-
mented through the extraction and combination of the global
characteristics of the crowd [61]–[63]. In this study, the theory
of personality traits can also be used to dynamically learn the
behavior of pedestrians in a given scene and to calculate a
model of movement for this pedestrian [64].
A widely used solution is the integration of the long
short-term memory (LSTM) algorithm [65] with the DRL.
In this type of solution, LSTM can be used to reflect the
memory characteristics of humans and accelerate the learning
of policies for autonomous agent navigation [66]–[69].
An LSTM-LMC architecture using LSTM and Local-Map
Critic (LMC) was proposed for autonomous navigation in
complex environments with a limited field of view [70]. The
solution used a map with static obstacles to reinforce the
learning ability, combined with a dataset containing 75 maps,
randomly sampled at each training episode. To overcome
application performance failures in real scenarios and improve
the robustness of the learned policy, different randomization
techniques were applied during tests in a simulated environ-
ment.
The construction of a navigation policy for unknown dy-
namic environments using a multiagent approach applied DRL
by combining LSTM with the PPO and RVO algorithms [71].
The PPO algorithm based on actor/critic method was used to
train agents to learn on how to achieve their goals, whereas the
RVO was used to avoid collisions during the navigation task.
The tests showed that the proposed solution could simultane-
ously train multiagents with different objectives, obtain good
results during path planning, and present a good capacity for
self-learning. Further, the authors highlighted that LSTM can
address two common problems that may occur when training
traditional recurrent neural networks—the vanishing gradient
problem [72] and the exploding gradient problem [73].
B. Based on Trajectory Prediction
Studies that focusing on the planning of trajectories in envi-
ronments involving everyday situations have highlighted that
humans do not have the habit of following shorter trajectories
when navigating crowds [74]. In this type of environment,
social norms are generally considered when implementing
mechanisms such that the robot avoids the most dense areas
with high probability of collisions.
To obtain viable trajectories, in addition to predicting the
future evolution of obstacles, the need to observe kinodynamic
restrictions of the robot has also been highlighted based on
the safety, maneuverability, and restrictions of the environment
[75]. In a complementary manner, the agent should be able to
learn an interaction model from the trajectory data [76], [77]
or real human behavior, model the velocities of other agents
in the crowd, or identify the variable personality at of each
pedestrian [78].
Some solutions use agent-based methods to predict pedes-
trian trajectories using velocity-space reasoning [79]. In addi-
tion to not relying on prior knowledge of the environment, this
type of approach can be integrated with other local navigation
techniques to improve the rate of task completion and reduce
instances related to the problem of robot freezing.
Fig. 5. Dynamic Channel
Dynamic channels architecture for autonomous crowd nav-
igation (Fig. 5) combines global optimization with a heuristic
to manage agent dynamics in the crowd [80]. The proposed
142
Authorized licensed use limited to: FUDAN UNIVERSITY. Downloaded on February 25,2023 at 04:33:29 UTC from IEEE Xplore. Restrictions apply.
method uses a modification of the algorithm A* to calculate
the ideal path, abstracting the environment through the trian-
gulation of Delaunay [81], and then projecting the dynamics
of the obstacles in the triangulation.
The solution presented did not explicitly address socially
acceptable navigation. However, it can be extended to incor-
porate social norms, such as a person taking pictures of others
or a group of friends walking together. Therefore, the edges
between pedestrians can be denoted as noncrossable for the
robot.
V. DISCUSSION
This section presents a brief analysis of the studies selected
for this review. Among the ones selected, 55% of the studies
used the reactive approach to prevent collisions during the
navigation task, and 45% used predictive methods. A signifi-
cant rise in the use of learning-based techniques, particularly
after 2017, corresponding to 61% of the selected studies, was
noted. Additionally, 33% of the studies addressed the use of
techniques aimed at social navigation, with significant progress
in this type of solutions from 2021.
The analysis results also revealed that there is a significant
increase in the proposition of studies that focused on social
navigation and use of DRL for navigation in crowded envi-
ronments; it significant studies demonstrates the need to adopt
policies for collision prevention and a combine techniques that
provide efficient navigation without endangering people close
to the robot.
Therefore, for autonomous navigation in dynamic and
crowded environments, three essential points should be ob-
served: a) the need for a collision prevention policy through
reactive or predictive behaviors; b) the ability to promptly
calculate a viable trajectory and change the trajectory of the
robot based on updated information of the environment; and
c) the ability to identify people to improve decision-making
and perform the task of navigation more safely.
VI. CONCLUSIONS
This study presents a literature review on addressing au-
tonomous navigation in crowded indoor environments. We
considered studies in this field that were published in the
last seven years. The analysis showed that the navigation of
autonomous robots in crowded environments remains an open
problem.
The existing approaches disregard the number of agents as
a dynamic variable or consider the trajectory or velocity of
the other agents to be known. However, some studies have
shown that in this type of scenarios, the robot should be able
to navigate without prior knowledge of the environment or
actions of other agents and obstacles.
It was also noted that most of the proposed solutions
consider people as simple moving obstacles, which can affect
the safety of navigation in real-world applications. Therefore,
it is possible to follow a line of studies in this field to
address new solutions, particularly through a combination of
techniques, or investigating and improving existing solutions
to efficiently deal address the problem in question.
 R EFERENCES
[1] J. Godoy, S. J. Guy, M. Gini, and I. Karamouzas, “C-nav: Distributed
coordination in crowded multi-agent navigation,” Robotics and Au-
tonomous Systems, vol. 133, p. 103631, 2020.
[2] M. Nishimura and R. Yonetani, “L2b: Learning to balance the safety-
efficiency trade-off in interactive crowd-aware robot navigation,” in 2020
IEEE/RSJ International Conference on Intelligent Robots and Systems
(IROS), 2020, pp. 11 004–11 010.
[3] D. Dugas, J. Nieto, R. Siegwart, and J. J. Chung, “Ian: Multi-behavior
navigation planning for robots in real, crowded environments,” in 2020
IEEE/RSJ International Conference on Intelligent Robots and Systems
(IROS), 2020, pp. 11 368–11 375.
[4] Y. Chen, C. Liu, B. E. Shi, and M. Liu, “Robot navigation in crowds by
graph convolutional networks with attention learned from human gaze,”
IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 2754–2761,
2020.
[5] V. N. Zhidkov, N. V. Kim, and N. V. Udalova, “Robot navigation
in a dynamically changing environment,” in 2020 International Multi-
Conference on Industrial Engineering and Modern Technologies (Far-
EastCon), 2020, pp. 1–6.
[6] D. Zhang, Z. Xie, P. Li, J. Yu, and X. Chen, “Real-time navigation
in dynamic human environments using optimal reciprocal collision
avoidance,” in 2015 IEEE International Conference on Mechatronics
and Automation (ICMA). IEEE, 2015, pp. 2232–2237.
[7] D. J. Gonon, D. Paez-Granados, and A. Billard, “Reactive navigation in
crowds for non-holonomic robots with convex bounding shape,” IEEE
Robotics and Automation Letters, vol. 6, no. 3, pp. 4728–4735, 2021.
[8] M. Pfeiffer, U. Schwesinger, H. Sommer, E. Galceran, and R. Siegwart,
“Predicting actions to act predictably: Cooperative partial motion plan-
ning with maximum entropy models,” in 2016 IEEE/RSJ International
Conference on Intelligent Robots and Systems (IROS). IEEE, 2016, pp.
2096–2101.
[9] Y. Xu, D. Ren, M. Li, Y. Chen, M. Fan, and H. Xia, “Tra2tra: Trajectory-
to-trajectory prediction with a global social spatial-temporal attentive
neural network,” IEEE Robotics and Automation Letters, vol. 6, no. 2,
pp. 1574–1581, 2021.
[10] Y. Cui, H. Zhang, Y. Wang, and R. Xiong, “Learning world transition
model for socially aware robot navigation,” in 2021 IEEE International
Conference on Robotics and Automation (ICRA), 2021, pp. 9262–9268.
[11] T. Fan, X. Cheng, J. Pan, D. Monacha, and R. Yang, “Crowdmove:
Autonomous mapless navigation in crowded scenarios,” arXiv preprint
arXiv:1807.07870, 2018.
[12] Y. F. Chen, M. Everett, M. Liu, and J. P. How, “Socially aware
motion planning with deep reinforcement learning,” in 2017 IEEE/RSJ
International Conference on Intelligent Robots and Systems (IROS).
IEEE, 2017, pp. 1343–1350.
[13] S. S. Samsani and M. S. Muhammad, “Socially compliant robot navi-
gation in crowded environment by human behavior resemblance using
deep reinforcement learning,” IEEE Robotics and Automation Letters,
vol. 6, no. 3, pp. 5223–5230, 2021.
[14] D. Bareiss and J. van den Berg, “Generalized reciprocal collision
avoidance,” The International Journal of Robotics Research, vol. 34,
no. 12, pp. 1501–1514, 2015.
[15] F. d. A. M. Pimentel and P. T. Aquino-Jr, “Evaluation of ros navigation
stack for social navigation in simulated environments,” Journal of
Intelligent & Robotic Systems, vol. 102, no. 4, pp. 1–18, 2021.
[16] M. Daza, D. Barrios-Aranibar, J. Diaz-Amado, Y. Cardinale, and J. Vi-
lasboas, “An approach of social navigation based on proxemics for
crowded environments of humans and robots,” Micromachines, vol. 12,
no. 2, p. 193, 2021.
[17] H. Zeng, R. Hu, X. Huang, and Z. Peng, “Robot navigation in crowd
based on dual social attention deep reinforcement learning,” Mathemat-
ical Problems in Engineering, vol. 2021, 2021.
[18] F. Haarslev., W. Juel., A. Kollakidou., N. Krüger., and L. Bodenhagen.,
“Context-aware social robot navigation,” in Proceedings of the 18th
International Conference on Informatics in Control, Automation and
Robotics - ICINCO,, INSTICC. SciTePress, 2021, pp. 426–433.
[19] Y. Liu, H. Liu, and B. Wang, “Autonomous exploration for mobile robot
using q-learning,” in 2017 2nd International Conference on Advanced
Robotics and Mechatronics (ICARM). IEEE, 2017, pp. 614–619.
[20] L. Tai and M. Liu, “A robot exploration strategy based on q-learning net-
work,” in 2016 IEEE International Conference on Real-time Computing
and Robotics (RCAR). IEEE, 2016, pp. 57–62.
143
Authorized licensed use limited to: FUDAN UNIVERSITY. Downloaded on February 25,2023 at 04:33:29 UTC from IEEE Xplore. Restrictions apply.
[21] R. S. Sutton, “Introduction: The challenge of reinforcement learning,”
in Reinforcement Learning. Springer, 1992, pp. 1–3.
[22] ——, “Reinforcement learning: Past, present and future,” in Asia-Pacific
Conference on Simulated Evolution and Learning. Springer, 1998, pp.
195–197.
[23] L. Qiang, D. Nanxun, L. Huican, and W. Heng, “A model-free mapless
navigation method for mobile robot using reinforcement learning,” in
2018 Chinese Control And Decision Conference (CCDC). IEEE, 2018,
pp. 3410–3415.
[24] C. J. Watkins and P. Dayan, “Q-learning,” Machine learning, vol. 8, no.
3-4, pp. 279–292, 1992.
[25] T. Ribeiro, F. Gonçalves, I. Garcia, G. Lopes, and A. F. Ribeiro, “Q-
learning for autonomous mobile robot obstacle avoidance,” in 2019
IEEE International Conference on Autonomous Robot Systems and
Competitions (ICARSC). IEEE, 2019, pp. 1–7.
[26] S. Yang and C. Li, “Behavior control algorithm for mobile robot based
on q-learning,” in 2017 International Conference on Computer Network,
Electronic and Automation (ICCNEA). IEEE, 2017, pp. 45–48.
[27] D. Zhang, X. Han, and C. Deng, “Review on the research and practice of
deep learning and reinforcement learning in smart grids,” CSEE Journal
of Power and Energy Systems, vol. 4, no. 3, pp. 362–370, 2018.
[28] L. P. Kaelbling, M. L. Littman, and A. W. Moore, “Reinforcement
learning: A survey,” Journal of artificial intelligence research, vol. 4,
pp. 237–285, 1996.
[29] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G.
Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski,
et al., “Human-level control through deep reinforcement learning,”
Nature, vol. 518, no. 7540, p. 529, 2015.
[30] L. Tai and M. Liu, “Towards cognitive exploration through deep rein-
forcement learning for mobile robots,” arXiv preprint arXiv:1610.01733,
2016.
[31] T. Okuyama, T. Gonsalves, and J. Upadhay, “Autonomous driving
system based on deep q learnig,” in 2018 International Conference on
Intelligent Autonomous Systems (ICoIAS). IEEE, 2018, pp. 201–205.
[32] A. E. Sallab, M. Abdou, E. Perot, and S. Yogamani, “Deep reinforcement
learning framework for autonomous driving,” Electronic Imaging, vol.
2017, no. 19, pp. 70–76, 2017.
[33] M. Wulfmeier, D. Rao, D. Z. Wang, P. Ondruska, and I. Posner,
“Large-scale cost function learning for path planning using deep inverse
reinforcement learning,” The International Journal of Robotics Research,
vol. 36, no. 10, pp. 1073–1087, 2017.
[34] G. Zuo, T. Du, and J. Lu, “Double dqn method for object detection,”
in 2017 Chinese Automation Congress (CAC). IEEE, 2017, pp. 6727–
6732.
[35] P. K. Mohanty, A. K. Sah, V. Kumar, and S. Kundu, “Application of deep
q-learning for wheel mobile robot navigation,” in 2017 3rd International
Conference on Computational Intelligence and Networks (CINE). IEEE,
2017, pp. 88–93.
[36] X. Ruan, D. Ren, X. Zhu, and J. Huang, “Mobile robot navigation based
on deep reinforcement learning,” in 2019 Chinese Control And Decision
Conference (CCDC). IEEE, 2019, pp. 6174–6178.
[37] H. T. Le, D. T. Nguyen, and X. T. Truong, “Socially aware robot navi-
gation framework in crowded and dynamic environments: A comparison
of motion planning techniques,” in 2021 8th NAFOSTED Conference on
Information and Computer Science (NICS), 2021, pp. 95–101.
[38] L. Liu, D. Dugas, G. Cesari, R. Siegwart, and R. Dubé, “Robot
navigation in crowded environments using deep reinforcement learning,”
in 2020 IEEE/RSJ International Conference on Intelligent Robots and
Systems (IROS), 2020, pp. 5671–5677.
[39] Y. F. Chen, M. Liu, M. Everett, and J. P. How, “Decentralized non-
communicating multiagent collision avoidance with deep reinforcement
learning,” in 2017 IEEE international conference on robotics and
automation (ICRA). IEEE, 2017, pp. 285–292.
[40] Y. Zhai, Y. Miao, and H. Wang, “Robot navigation with interaction-based
deep reinforcement learning,” in 2021 IEEE International Conference on
Robotics and Biomimetics (ROBIO), 2021, pp. 1974–1979.
[41] M. Everett, Y. F. Chen, and J. P. How, “Motion planning among dynamic,
decision-making agents with deep reinforcement learning,” in 2018
IEEE/RSJ International Conference on Intelligent Robots and Systems
(IROS). IEEE, 2018, pp. 3052–3059.
[42] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox-
imal policy optimization algorithms,” arXiv preprint arXiv:1707.06347,
2017.
[43] P. Long, T. Fanl, X. Liao, W. Liu, H. Zhang, and J. Pan, “Towards
optimally decentralized multi-robot collision avoidance via deep rein-
forcement learning,” in 2018 IEEE International Conference on Robotics
and Automation (ICRA). IEEE, 2018, pp. 6252–6259.
[44] S. Yao, G. Chen, Q. Qiu, J. Ma, X. Chen, and J. Ji, “Crowd-aware robot
navigation for pedestrians with multiple collision avoidance strategies
via map-based deep reinforcement learning,” in 2021 IEEE/RSJ Inter-
national Conference on Intelligent Robots and Systems (IROS), 2021,
pp. 8144–8150.
[45] Y. Liu, A. Xu, and Z. Chen, “Map-based deep imitation learning for
obstacle avoidance,” in 2018 IEEE/RSJ International Conference on
Intelligent Robots and Systems (IROS). IEEE, 2018, pp. 8644–8649.
[46] P. Long, W. Liu, and J. Pan, “Deep-learned collision avoidance policy
for distributed multiagent navigation,” IEEE Robotics and Automation
Letters, vol. 2, no. 2, pp. 656–663, 2017.
[47] J. Wu, S. Shin, C.-G. Kim, and S.-D. Kim, “Effective lazy training
method for deep q-network in obstacle avoidance and path planning,” in
2017 IEEE International Conference on Systems, Man, and Cybernetics
(SMC). IEEE, 2017, pp. 1799–1804.
[48] P. Wu, S. Xie, H. Liu, J. Luo, Q. Li, and J. Gu, “A novel obstacle avoid-
ance strategy of nonholonomic mobile robot based on virtual simulation
platform,” in 2015 IEEE International Conference on Information and
Automation. IEEE, 2015, pp. 185–190.
[49] D. Lee, J. Jeong, Y. H. Kim, and J. B. Park, “An improved artificial
potential field method with a new point of attractive force for a
mobile robot,” in 2017 2nd International Conference on Robotics and
Automation Engineering (ICRAE). IEEE, 2017, pp. 63–67.
[50] H.-T. Chiang, N. Malone, K. Lesser, M. Oishi, and L. Tapia, “Path-
guided artificial potential fields with stochastic reachable sets for motion
planning in highly dynamic environments,” in 2015 IEEE International
Conference on Robotics and Automation (ICRA). IEEE, 2015, pp.
2347–2354.
[51] N. Malone, H.-T. Chiang, K. Lesser, M. Oishi, and L. Tapia, “Hybrid
dynamic moving obstacle avoidance using a stochastic reachable set-
based potential field,” IEEE Transactions on Robotics, vol. 33, no. 5,
pp. 1124–1138, 2017.
[52] X. Gu, M. Han, W. Zhang, G. Xue, G. Zhang, and Y. Han, “Intelli-
gent vehicle path planning based on improved artificial potential field
algorithm,” in 2019 International Conference on High Performance Big
Data and Intelligent Systems (HPBD&IS). IEEE, 2019, pp. 104–109.
[53] P. Fiorini and Z. Shiller, “Motion planning in dynamic environments us-
ing velocity obstacles,” The International Journal of Robotics Research,
vol. 17, no. 7, pp. 760–772, 1998.
[54] J. A. Douthwaite, S. Zhao, and L. S. Mihaylova, “A comparative study of
velocity obstacle approaches for multi-agent systems,” in 2018 UKACC
12th International Conference on Control (CONTROL). IEEE, 2018,
pp. 289–294.
[55] J. Van den Berg, M. Lin, and D. Manocha, “Reciprocal velocity obsta-
cles for real-time multi-agent navigation,” in 2008 IEEE International
Conference on Robotics and Automation. IEEE, 2008, pp. 1928–1935.
[56] J. Snape, J. Van Den Berg, S. J. Guy, and D. Manocha, “The hybrid
reciprocal velocity obstacle,” IEEE Transactions on Robotics, vol. 27,
no. 4, pp. 696–706, 2011.
[57] J. Van Den Berg, S. J. Guy, M. Lin, and D. Manocha, “Reciprocal n-
body collision avoidance,” in Robotics research. Springer, 2011, pp.
3–19.
[58] Z. Gyenes and E. G. Szadeczky-Kardoss, “Motion planning for mobile
robots using the safety velocity obstacles method,” in 2018 19th In-
ternational Carpathian Control Conference (ICCC). IEEE, 2018, pp.
389–394.
[59] M.-T. Lorente, E. Owen, and L. Montano, “Model-based robocentric
planning and navigation for dynamic environments,” The International
Journal of Robotics Research, vol. 37, no. 8, pp. 867–889, 2018.
[60] G. Ferrer and A. Sanfeliu, “Anticipative kinodynamic planning: multi-
objective robot navigation in urban and dynamic environments,” Au-
tonomous Robots, pp. 1–16, 2018.
[61] P.-H. Ciou, Y.-T. Hsiao, Z.-Z. Wu, S.-H. Tseng, and L.-C. Fu, “Compos-
ite reinforcement learning for social robot navigation,” in 2018 IEEE/RSJ
International Conference on Intelligent Robots and Systems (IROS).
IEEE, 2018, pp. 2553–2558.
[62] S. H. Kiss, K. Katuwandeniya, A. Alempijevic, and T. Vidal-Calleja,
“Probabilistic dynamic crowd prediction for social navigation,” in 2021
IEEE International Conference on Robotics and Automation (ICRA),
2021, pp. 9269–9275.
144
Authorized licensed use limited to: FUDAN UNIVERSITY. Downloaded on February 25,2023 at 04:33:29 UTC from IEEE Xplore. Restrictions apply.
[63] S. Matsuzaki, S. Aonuma, and Y. Hasegawa, “Dynamic window ap-
proach with human imitating collision avoidance,” in 2021 IEEE In-
ternational Conference on Robotics and Automation (ICRA), 2021, pp.
8180–8186.
[64] A. Bera, T. Randhavane, and D. Manocha, “Aggressive, tense or shy?
identifying personality traits from crowd videos.” in IJCAI, 2017, pp.
112–118.
[65] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural
computation, vol. 9, no. 8, pp. 1735–1780, 1997.
[66] A. Alahi, K. Goel, V. Ramanathan, A. Robicquet, L. Fei-Fei, and
S. Savarese, “Social lstm: Human trajectory prediction in crowded
spaces,” in Proceedings of the IEEE conference on computer vision and
pattern recognition, 2016, pp. 961–971.
[67] M. Lisotto, P. Coscia, and L. Ballan, “Social and scene-aware trajectory
prediction in crowded spaces,” in 2019 IEEE/CVF International Con-
ference on Computer Vision Workshop (ICCVW), 2019, pp. 2567–2574.
[68] A. Gupta, J. Johnson, L. Fei-Fei, S. Savarese, and A. Alahi, “Social gan:
Socially acceptable trajectories with generative adversarial networks,” in
Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition, 2018, pp. 2255–2264.
[69] M. Pfeiffer, G. Paolo, H. Sommer, J. Nieto, R. Siegwart, and C. Cadena,
“A data-driven model for interaction-aware pedestrian motion prediction
in object cluttered environments,” in 2018 IEEE International Confer-
ence on Robotics and Automation (ICRA). IEEE, 2018, pp. 1–8.
[70] J. Choi, K. Park, M. Kim, and S. Seok, “Deep reinforcement learning of
navigation in a complex and crowded environment with a limited field
of view,” in 2019 International Conference on Robotics and Automation
(ICRA), 2019, pp. 5993–6000.
[71] L. Sun, J. Zhai, and W. Qin, “Crowd navigation in an unknown and
dynamic environment based on deep reinforcement learning,” IEEE
Access, vol. 7, pp. 109 544–109 554, 2019.
[72] S. Hochreiter, “The vanishing gradient problem during learning recurrent
neural nets and problem solutions,” International Journal of Uncertainty,
Fuzziness and Knowledge-Based Systems, vol. 6, no. 02, pp. 107–116,
1998.
[73] R. Pascanu, T. Mikolov, and Y. Bengio, “On the difficulty of training
recurrent neural networks,” in International conference on machine
learning. PMLR, 2013, pp. 1310–1318.
[74] R. Bresson, J. Saraydaryan, J. Dugdale, and A. Spalanzani, “Socially
compliant navigation in dense crowds,” in 2019 IEEE Intelligent Vehicles
Symposium (IV), 2019, pp. 64–69.
[75] A. Vemula, K. Muelling, and J. Oh, “Modeling cooperative navigation in
dense human crowds,” in Robotics and Automation (ICRA), 2017 IEEE
International Conference on. IEEE, 2017, pp. 1685–1692.
[76] M. Fahad, G. Yang, and Y. Guo, “Learning human navigation behav-
ior using measured human trajectories in crowded spaces,” in 2020
IEEE/RSJ International Conference on Intelligent Robots and Systems
(IROS), 2020, pp. 11 154–11 160.
[77] A. J. Sathyamoorthy, J. Liang, U. Patel, T. Guan, R. Chandra, and
D. Manocha, “Densecavoid: Real-time navigation in dense crowds using
anticipatory behaviors,” in 2020 IEEE International Conference on
Robotics and Automation (ICRA), 2020, pp. 11 345–11 352.
[78] A. Bera, T. Randhavane, R. Prinja, and D. Manocha, “Sociosense:
Robot navigation amongst pedestrians with social and psychological
constraints,” in Intelligent Robots and Systems (IROS), 2017 IEEE/RSJ
International Conference on. IEEE, 2017, pp. 7018–7025.
[79] S. Kim, S. J. Guy, W. Liu, D. Wilkie, R. W. Lau, M. C. Lin, and
D. Manocha, “Brvo: Predicting pedestrian trajectories using velocity-
space reasoning,” The International Journal of Robotics Research,
vol. 34, no. 2, pp. 201–217, 2015.
[80] C. Cao, P. Trautman, and S. Iba, “Dynamic channel: A planning
framework for crowd navigation,” in 2019 International Conference on
Robotics and Automation (ICRA), 2019, pp. 5551–5557.
[81] L. P. Chew, “Constrained delaunay triangulations,” Algorithmica, vol. 4,
no. 1, pp. 97–108, 1989.