Abstract:How to use environmental information to improve the cooperative chasing efficiency of multiple unmanned surface vehicles (multi-USV) is a problem worthy of attention. This article considers an archipelago scenario and proposes an obstacle-assisted chasing framework based on reinforcement learning, in which a multi-USV system chases a smart evader through autonomous environmental awareness and target state prediction. When there are obstacles near the evader, the chasing group can use them to quickly complete the chase. Otherwise, within the range of perception, the group can drive the evader closer to the obstacles or directly cooperate to catch the evader. At the same time, the problem of group credit allocation is also considered, so as to achieve a relatively stable encircling structure with the least number of vehicles. Simulation experiments in the multi-obstacle environment demonstrate the flexibility of the proposed framework. Compared with the traditional force-based and learning-based methods in untrained archipelago environments, the model trained by this framework is proven to have higher efficiency while ensuring real-time performance and generalization. Note to Practitioners —The motivation of this paper is to provide guidance to the USV team in learning how to collaboratively chase and capture target through multi-agent reinforcement learning. Meanwhile, this method can also be applied to other chasing scenarios for vehicles, such as ground robots. This paper proposes a new method that leverages the MA-POCA learning framework and RSA mechanism to enhance cooperation and credit allocation among USVs. Additionally, a novel reward mechanism is designed to encourage USVs to fully utilize the advantages of the surrounding environment and accelerate cooperative chasing. With this method, multiple USVs can effectively pursue intelligent evaders through autonomous environmental perception and target state prediction. To validate the proposed method, we developed a high-fidelity simulation environment using Unity3D, incorporating USV dynamics, irregular obstacles, and sea surface disturbances. Furthermore, our method demonstrates low computational cost, making it suitable for practical USV navigation and control applications in the future.

Game of Marine Robots: USV Pursuit Evasion Game Using Online Reinforcement Learning

Pursuit-Evasion Game of Multiple Unmannd Surface Vessels in Partially Observable Environments

Pursuit-Evasion Game of Unmanded Surface Vehicles Based on Deep Reinforcement Learning

Large Scale Pursuit-Evasion under Collision Avoidance Using Deep Reinforcement Learning.

Integral Reinforcement Learning Based Dynamic Stackelberg Pursuit-Evasion Game for Unmanned Surface Vehicles

Pursuit-evasion Game Strategy of USV Based on Deep Reinforcement Learning in Complex Multi-Obstacle Environment

An Improved Approach Towards Multi-Agent Pursuit–Evasion Game Decision-Making Using Deep Reinforcement Learning

Pursuit-Evasion Games for Multi-agent Based on Reinforcement Learning with Obstacles

Multi-AUV Pursuit-Evasion Game in the Internet of Underwater Things: an Efficient Training Framework Via Offline Reinforcement Learning

Pursuit and evasion game between UVAs based on multi-agent reinforcement learning

Multi-UAV Pursuit-Evasion with Online Planning in Unknown Environments by Deep Reinforcement Learning

Distributed Pursuit-Evasion Game of Limited Perception USV Swarm Based on Multiagent Proximal Policy Optimization

Pursuit-Evasion Games of Marine Surface Vessels Using Neural Network-Based Control

Evade Unknown Pursuer Via Pursuit Strategy Identification and Model Reference Policy Adaptation (MRPA) Algorithm

Research on Intelligent Evasion Methods for UAV Based on Deep Reinforcement Learning

An Application of Continuous Deep Reinforcement Learning Approach to Pursuit-Evasion Differential Game

Multi-USV Cooperative Chasing Strategy Based on Obstacles Assistance and Deep Reinforcement Learning

Differential Game-Based Deep Reinforcement Learning in Underwater Target Hunting Task

A Reinforcement Learning Solution to the Nonlinear Spacecraft Pursuit-Evasion Game Problem

A Novel Reinforcement Learning Collision Avoidance Algorithm for USVs Based on Maneuvering Characteristics and COLREGs

Threat Potential Field Based Pursuit–Evasion Games for Underactuated Unmanned Surface Vehicles