Abstract:To solve the problem of multi-target hunting by an unmanned surface vehicle (USV) fleet, a hunting algorithm based on multi-agent reinforcement learning is proposed. Firstly, the hunting environment and kinematic model without boundary constraints are built, and the criteria for successful target capture are given. Then, the cooperative hunting problem of a USV fleet is modeled as a decentralized partially observable Markov decision process (Dec-POMDP), and a distributed partially observable multi-target hunting Proximal Policy Optimization (DPOMH-PPO) algorithm applicable to USVs is proposed. In addition, an observation model, a reward function and the action space applicable to multi-target hunting tasks are designed. To deal with the dynamic change of observational feature dimension input by partially observable systems, a feature embedding block is proposed. By combining the two feature compression methods of column-wise max pooling (CMP) and column-wise average-pooling (CAP), observational feature encoding is established. Finally, the centralized training and decentralized execution framework is adopted to complete the training of hunting strategy. Each USV in the fleet shares the same policy and perform actions independently. Simulation experiments have verified the effectiveness of the DPOMH-PPO algorithm in the test scenarios with different numbers of USVs. Moreover, the advantages of the proposed model are comprehensively analyzed from the aspects of algorithm performance, migration effect in task scenarios and self-organization capability after being damaged, the potential deployment and application of DPOMH-PPO in the real environment is verified.

Safe Multi-Agent Learning Control for Unmanned Surface Vessels Cooperative Interception Mission

Safe deep reinforcement learning-based adaptive control for USV interception mission

Model Predictive Control Based on State Space and Risk Augmentation for Unmanned Surface Vessel Trajectory Tracking

Cooperative Target Enclosing Control for Multiple Unmanned Surface Vehicles with Unknown Dynamics and Safety Assurance

Multi-USV Dynamic Navigation and Target Capture: A Guided Multi-Agent Reinforcement Learning Approach

Safe Multiagent Learning with Soft Constrained Policy Optimization in Real Robot Control

Proximal policy optimization with reciprocal velocity obstacle based collision avoidance path planning for multi-unmanned surface vehicles

Cooperative Multi-Target Hunting by Unmanned Surface Vehicles Based on Multi-Agent Reinforcement Learning

Multi-USV Deep Reinforcement Learning for Distributed Cooperative Target Tracking

Research on the Multiagent Joint Proximal Policy Optimization Algorithm Controlling Cooperative Fixed-Wing UAV Obstacle Avoidance

Dynamic Navigation and Area Assignment of Multiple USVs Based on Multi-Agent Deep Reinforcement Learning

Collision Avoidance Control for Limited Perception Unmanned Surface Vehicle Swarm Based on Proximal Policy Optimization

Proximal Policy Optimization with Proportional-Differential Feedback for Tracking Control of Unmanned Surface Vessel

An Iterative Learning-based Integrated Motion Planning and Control Method for Autonomous Patrolling of Unmanned Surface Vehicles

Deep Reinforcement Learning Based Multi-UUV Cooperative Control for Target Capturing

Distributed Model Predictive Contouring Control of Unmanned Surface Vessels

Multi-USV System Antidisturbance Cooperative Searching Based on the Reinforcement Learning Method

Distributional Soft Actor-Critic-Based Multi-AUV Cooperative Pursuit for Maritime Security Protection

Cooperative Path Following Control of Unmanned Surface Vehicles Using Model Predictive Control

Intelligent Decision-Making System for Multiple Marine Autonomous Surface Ships Based on Deep Reinforcement Learning

Multi-USV System Cooperative Underwater Target Search Based on Reinforcement Learning and Probability Map