Abstract:Formation maintenance and collision avoidance allow multiple agents to form and maintain a specific formation in the environment and reach the final goal safely without any collisions, which are important component of the multi-agent/crowd cooperation field. We propose a two-level formation control method to simulate crowd formation. At the low level, a comprehensive group reward function is designed to train the agents to learn to collaborate to reach the goal and avoid collisions with other agents and obstacles. The Multi-Agent POsthumous Credit Assignment algorithm and curriculum learning are combined to solve the credit assignment problem in the shared group reward and the slow convergence problem of the policy in the reward-sparse environment. At the high level, we design a formation switch method to determine the compression or splitting of the formation for multiple agents passing through the narrow exit. Then the optimal reciprocal collision avoidance algorithm is adopted to compute a collision-free velocity for the team of multiple agents, on the basis of which the desired formation positions are calculated to avoid collisions between the team and the obstacles in the environment. Lastly, Hungarian algorithm is utilized to assign the formation positions to the agents as their respective target position to accelerate the process of forming. Experimental results demonstrate that our proposed approach enable multiple agents to constitute the formation shape from the starting positions quickly and maintain the shape well when moving toward the final goal. Furthermore, our model has a certain generalization ability to unseen formation shapes in the scenarios without and with obstacles.

Adaptive Formation Tracking of Swarm Jumping Robots Using Multiagent Deep Reinforcement Learning

Cooperative Flocking And Learning In Multi-Robot Systems For Predator Avoidance

Moving Forward in Formation: A Decentralized Hierarchical Learning Approach to Multi-Agent Moving Together

Socially-Aware Multi-Agent Following with 2D Laser Scans Via Deep Reinforcement Learning and Potential Field

Mapless Collaborative Navigation for a Multi-Robot System Based on the Deep Reinforcement Learning

Multi-UAV Adaptive Cooperative Formation Trajectory Planning Based on an Improved MATD3 Algorithm of Deep Reinforcement Learning

Underwater Multi-agent Cooperative Formation Hunting Based on Deep Reinforcement Learning

Obstacle Avoidance in Multi-Agent Formation Process Based on Deep Reinforcement Learning

Learning-Based Multi-Robot Formation Control With Obstacle Avoidance

The crowd cooperation approach for formation maintenance and collision avoidance using multi-agent deep reinforcement learning

Reinforcement learning for multi-agent formation navigation with scalability

Relative Distributed Formation and Obstacle Avoidance with Multi-agent Reinforcement Learning

Self-Organizing and Scalable Pattern Formation in Swarming Systems Using Lennard-Jones Potential Field

Adaptive Distributed Cooperative Tracking Control and Application for Multi-agent Formation under Communication Constraints

Event-triggered Formation Control with Obstacle Avoidance for Multi-Agent Systems Applied to Multi-Uav Formation Flying

A Multiobjective Collaborative Deep Reinforcement Learning Algorithm for Jumping Optimization of Bipedal Robot

An Attention Mechanism and Adaptive Accuracy Triple-Dependent MADDPG Formation Control Method for Hybrid UAVs

Sim-real joint experimental verification for an unmanned surface vehicle formation strategy based on multi-agent deterministic policy gradient and line of sight guidance

Improving multi-target cooperative tracking guidance for UAV swarms using multi-agent reinforcement learning

Leader-Following Formation Tracking of Multiagent Systems Using Adaptive Scaling Mechanism under Spatial Constraints

Autonomous Swarm Robot Coordination via Mean-Field Control Embedding Multi-Agent Reinforcement Learning