Abstract:To solve the problem of multi-target hunting by an unmanned surface vehicle (USV) fleet, a hunting algorithm based on multi-agent reinforcement learning is proposed. Firstly, the hunting environment and kinematic model without boundary constraints are built, and the criteria for successful target capture are given. Then, the cooperative hunting problem of a USV fleet is modeled as a decentralized partially observable Markov decision process (Dec-POMDP), and a distributed partially observable multi-target hunting Proximal Policy Optimization (DPOMH-PPO) algorithm applicable to USVs is proposed. In addition, an observation model, a reward function and the action space applicable to multi-target hunting tasks are designed. To deal with the dynamic change of observational feature dimension input by partially observable systems, a feature embedding block is proposed. By combining the two feature compression methods of column-wise max pooling (CMP) and column-wise average-pooling (CAP), observational feature encoding is established. Finally, the centralized training and decentralized execution framework is adopted to complete the training of hunting strategy. Each USV in the fleet shares the same policy and perform actions independently. Simulation experiments have verified the effectiveness of the DPOMH-PPO algorithm in the test scenarios with different numbers of USVs. Moreover, the advantages of the proposed model are comprehensively analyzed from the aspects of algorithm performance, migration effect in task scenarios and self-organization capability after being damaged, the potential deployment and application of DPOMH-PPO in the real environment is verified.

Collaborative exploration of multiple unmanned surface vessels in complex areas based on PPO algorithm

USV Collision Avoidance Decision-Making Based on the Improved PPO Algorithm in Restricted Waters

A Hybrid Path Planning Algorithm for Unmanned Surface Vehicles in Complex Environment with Dynamic Obstacles.

Cooperative Path Planning with Priority Target Assignment and Collision Avoidance Guidance for Rescue Unmanned Surface Vehicles in a Complex Ocean Environment

An Efficient Autonomous Exploration Framework for Unmanned Surface Vehicles in Unknown Waters

Dynamic Area Coverage with Multi-USV in Fully Connected State Using Graph Theory

Cooperative Search Path Planning for Multiple Unmanned Surface Vehicles

USVs Path Planning for Maritime Search and Rescue Based on POS-DQN: Probability of Success-Deep Q-Network

Hybrid Algorithm-Based Full Coverage Search Approach with Multiple AUVs to Unknown Environments in Internet of Underwater Things

Cooperative Multi-Target Hunting by Unmanned Surface Vehicles Based on Multi-Agent Reinforcement Learning

Intelligent multi-task allocation and planning for multiple unmanned surface vehicles (USVs) using self-organising maps and fast marching method

Intelligent Navigation Method for Multiple Marine Autonomous Surface Ships Based on Improved PPO Algorithm

Multi-UUV/USV Adaptive Cooperative Search Using Online State Information

Research on Real-Time Collision Avoidance and Path Planning of Usvs in Multi-Obstacle Ships Environment

Proximal policy optimization with reciprocal velocity obstacle based collision avoidance path planning for multi-unmanned surface vehicles

A Cooperative Hunting Method for Multi-USV Based on the A* Algorithm in an Environment with Obstacles

AUV Dynamic Obstacle Avoidance Method Based on Improved PPO Algorithm

Multi-AUV Cooperative Search Method Based on Dynamic Optimal Coverage

Path planning for Multi-USV target coverage in complex environments

Maritime Search and Rescue Networking Based on Multi-Agent Cooperative Communication.

Cooperative survey of seabed ROIs using multiple USVs with coverage path planning