Abstract:It is challenging to avoid obstacles safely and efficiently for multiple robots of different shapes in distributed and communication-free scenarios, where robots do not communicate with each other and only sense other robots’ positions and obstacles around them. Most existing multi-robot collision avoidance systems either require communication between robots or require expensive movement data of other robots, like velocities, accelerations and paths. In this paper, we propose a map-based deep reinforcement learning approach for multi-robot collision avoidance in a distributed and communication-free environment. We use the egocentric local grid map of a robot to represent the environmental information around it including its shape and observable appearances of other robots and obstacles, which can be easily generated by using multiple sensors or sensor fusion. Then we apply the distributed proximal policy optimization (DPPO) algorithm to train a convolutional neural network that directly maps three frames of egocentric local grid maps and the robot’s relative local goal positions into low-level robot control commands. Compared to other methods, the map-based approach is more robust to noisy sensor data, does not require robots’ movement data and considers sizes and shapes of related robots, which make it to be more efficient and easier to be deployed to real robots. We first train the neural network in a specified simulator of multiple mobile robots using DPPO, where a multi-stage curriculum learning strategy for multiple scenarios is used to improve the performance. Then we deploy the trained model to real robots to perform collision avoidance in their navigation without tedious parameter tuning. We evaluate the approach with multiple scenarios both in the simulator and on four differential-drive mobile robots in the real world. Both qualitative and quantitative experiments show that our approach is efficient and outperforms existing DRL-based approaches in many indicators. We also conduct ablation studies showing the positive effects of using egocentric grid maps and multi-stage curriculum learning.

C3F: Constant Collaboration and Communication Framework for Graph-Representation Dynamic Multi-Robotic Systems

C^2:Co-design of Robots via Concurrent Networks Coupling Online and Offline Reinforcement Learning

C2: Co-design of Robots Via Concurrent-Network Coupling Online and Offline Reinforcement Learning

Meeting-Merging-Mission: A Multi-robot Coordinate Framework for Large-Scale Communication-Limited Exploration.

Cooperative Flocking And Learning In Multi-Robot Systems For Predator Avoidance

Mapless Collaborative Navigation for a Multi-Robot System Based on the Deep Reinforcement Learning

CRMRL: Collaborative Relationship Meta Reinforcement Learning for Effectively Adapting to Type Changes in Multi-Robotic System

A Distributed Competitive and Collaborative Coordination for Multirobot Systems

CLFR-M: Continual Learning Framework for Robots Via Human Feedback and Dynamic Memory

Enhancing cooperation by cognition differences and consistent representation in multi-agent reinforcement learning

Distributed Non-Communicating Multi-Robot Collision Avoidance Via Map-Based Deep Reinforcement Learning

Integrating Online Learning and Connectivity Maintenance for Communication-Aware Multi-Robot Coordination

Federated Deep Reinforcement Learning for RIS-Assisted Indoor Multi-Robot Communication Systems

Multirobot Coordination with Deep Reinforcement Learning in Complex Environments.

Wireless Multi-Robot Collaboration: Communications, Perception, Control and Planning

Towards Optimally Decentralized Multi-Robot Collision Avoidance Via Deep Reinforcement Learning

Hierarchical Consensus-Based Multi-Agent Reinforcement Learning for Multi-Robot Cooperation Tasks

Distributed deep reinforcement learning based on bi-objective framework for multi-robot formation

DESIGN OF SELF-ORGANIZING SYSTEMS USING MULTI-AGENT REINFORCEMENT LEARNING AND THE COMPROMISE DECISION SUPPORT PROBLEM CONSTRUCT

Multi-robot Social-aware Cooperative Planning in Pedestrian Environments Using Multi-agent Reinforcement Learning

Improving Fast Adaptation for Newcomers in Multi-Robot Reinforcement Learning System.