Abstract:Formation maintenance and collision avoidance allow multiple agents to form and maintain a specific formation in the environment and reach the final goal safely without any collisions, which are important component of the multi-agent/crowd cooperation field. We propose a two-level formation control method to simulate crowd formation. At the low level, a comprehensive group reward function is designed to train the agents to learn to collaborate to reach the goal and avoid collisions with other agents and obstacles. The Multi-Agent POsthumous Credit Assignment algorithm and curriculum learning are combined to solve the credit assignment problem in the shared group reward and the slow convergence problem of the policy in the reward-sparse environment. At the high level, we design a formation switch method to determine the compression or splitting of the formation for multiple agents passing through the narrow exit. Then the optimal reciprocal collision avoidance algorithm is adopted to compute a collision-free velocity for the team of multiple agents, on the basis of which the desired formation positions are calculated to avoid collisions between the team and the obstacles in the environment. Lastly, Hungarian algorithm is utilized to assign the formation positions to the agents as their respective target position to accelerate the process of forming. Experimental results demonstrate that our proposed approach enable multiple agents to constitute the formation shape from the starting positions quickly and maintain the shape well when moving toward the final goal. Furthermore, our model has a certain generalization ability to unseen formation shapes in the scenarios without and with obstacles.

Multi-Agent Defense Cooperative Model Based On Switching Formation

Towards a cooperative soccer team in RoboCup

Moving Forward in Formation: A Decentralized Hierarchical Learning Approach to Multi-Agent Moving Together

Adaptive algorithm for multi-agent learning optimal cooperative pursuit strategy based on Markov game

A muti-agent defensive strategy based on monte carlo method

A method of network attack-defense game and collaborative defense decision-making based on hierarchical multi-agent reinforcement learning

Dynamic Formation Planning and Control for Robot Soccer Game with Multi-Agent Reinforcement Learning and Behavioral Model

A Multi-agent Formation Control Algorithm in Interference Environment

Switching Learning-Based Cooperative Control with Its Application to Connected Automated Vehicles

Design and Analysis of Decentralized Interactive Cyber Defense Approach based on Multi-agent Coordination.

Simulation Model of Network Instruction and Defense Using Multi-Agent

Adversarial Decision-Making for Moving Target Defense: A Multi-Agent Markov Game and Reinforcement Learning Approach

Modeling and Analysis of the Decentralized Interactive Cyber Defense Approach

The crowd cooperation approach for formation maintenance and collision avoidance using multi-agent deep reinforcement learning

Underwater Multi-agent Cooperative Formation Hunting Based on Deep Reinforcement Learning

Cooperative Behavior Acquisition Based Modular Q Learning in Multi-Agent System

Study on Learning Algorithm of Transfer Reinforcement for Multi-Agent Formation Control

A Deep Reinforcement Learning-Based Method Applied for Solving Multi-Agent Defense and Attack Problems.

Matching-Based Capture-the-Flag Games for Multi-Agent Systems

Multi-Agent Formation Control With Obstacle Avoidance Based On Receding Horizon Strategy

Multi-Robot Pursuit in Parameterized Formation via Imitation Learning