Abstract:How to use environmental information to improve the cooperative chasing efficiency of multiple unmanned surface vehicles (multi-USV) is a problem worthy of attention. This article considers an archipelago scenario and proposes an obstacle-assisted chasing framework based on reinforcement learning, in which a multi-USV system chases a smart evader through autonomous environmental awareness and target state prediction. When there are obstacles near the evader, the chasing group can use them to quickly complete the chase. Otherwise, within the range of perception, the group can drive the evader closer to the obstacles or directly cooperate to catch the evader. At the same time, the problem of group credit allocation is also considered, so as to achieve a relatively stable encircling structure with the least number of vehicles. Simulation experiments in the multi-obstacle environment demonstrate the flexibility of the proposed framework. Compared with the traditional force-based and learning-based methods in untrained archipelago environments, the model trained by this framework is proven to have higher efficiency while ensuring real-time performance and generalization. Note to Practitioners —The motivation of this paper is to provide guidance to the USV team in learning how to collaboratively chase and capture target through multi-agent reinforcement learning. Meanwhile, this method can also be applied to other chasing scenarios for vehicles, such as ground robots. This paper proposes a new method that leverages the MA-POCA learning framework and RSA mechanism to enhance cooperation and credit allocation among USVs. Additionally, a novel reward mechanism is designed to encourage USVs to fully utilize the advantages of the surrounding environment and accelerate cooperative chasing. With this method, multiple USVs can effectively pursue intelligent evaders through autonomous environmental perception and target state prediction. To validate the proposed method, we developed a high-fidelity simulation environment using Unity3D, incorporating USV dynamics, irregular obstacles, and sea surface disturbances. Furthermore, our method demonstrates low computational cost, making it suitable for practical USV navigation and control applications in the future.

Multi-USV Deep Reinforcement Learning for Distributed Cooperative Target Tracking

A Confidence-based Allocation Approach for USV Trajectory Tracking Via Human-Robot Co-Driving

Multi-USV Dynamic Navigation and Target Capture: A Guided Multi-Agent Reinforcement Learning Approach

Deep Reinforcement Learning Based Multi-UUV Cooperative Control for Target Capturing

Cooperative Multi-Target Hunting by Unmanned Surface Vehicles Based on Multi-Agent Reinforcement Learning

Scalable-MADDPG-Based Cooperative Target Invasion for a Multi-USV System

Multi-Agent Reinforcement Learning for Unmanned Surface Vehicle Hunting Target

Multi-USV System Cooperative Underwater Target Search Based on Reinforcement Learning and Probability Map

Distributed Information Fusion Based Trajectory Tracking for USV and UAV Clusters Via Multi-Agent Deep Learning Approach

Multi-USV Cooperative Chasing Strategy Based on Obstacles Assistance and Deep Reinforcement Learning

Secure and Cooperative Target Tracking Via AUV Swarm - A Reinforcement Learning Approach.

Cooperative Target Enclosing Control for Multiple Unmanned Surface Vehicles with Unknown Dynamics and Safety Assurance

Dynamic Navigation and Area Assignment of Multiple USVs Based on Multi-Agent Deep Reinforcement Learning

Multi-UAV Adaptive Cooperative Formation Trajectory Planning Based on an Improved MATD3 Algorithm of Deep Reinforcement Learning

Multi-USVs Coordinated Detection in Marine Environment with Deep Reinforcement Learning.

Safe Multi-Agent Learning Control for Unmanned Surface Vessels Cooperative Interception Mission

Sim-real joint experimental verification for an unmanned surface vehicle formation strategy based on multi-agent deterministic policy gradient and line of sight guidance

Improving multi-target cooperative tracking guidance for UAV swarms using multi-agent reinforcement learning

Multi-USV System Antidisturbance Cooperative Searching Based on the Reinforcement Learning Method

Multi-UAV Autonomous Collaborative Target Tracking in Dynamic Environment Based on Multi-agent Reinforcement Learning

Cooperative Target Substitution Tracking Control of Multiple Unmanned Surface Vehicles with Substitute USVs