Abstract:To solve the problem of multi-target hunting by an unmanned surface vehicle (USV) fleet, a hunting algorithm based on multi-agent reinforcement learning is proposed. Firstly, the hunting environment and kinematic model without boundary constraints are built, and the criteria for successful target capture are given. Then, the cooperative hunting problem of a USV fleet is modeled as a decentralized partially observable Markov decision process (Dec-POMDP), and a distributed partially observable multi-target hunting Proximal Policy Optimization (DPOMH-PPO) algorithm applicable to USVs is proposed. In addition, an observation model, a reward function and the action space applicable to multi-target hunting tasks are designed. To deal with the dynamic change of observational feature dimension input by partially observable systems, a feature embedding block is proposed. By combining the two feature compression methods of column-wise max pooling (CMP) and column-wise average-pooling (CAP), observational feature encoding is established. Finally, the centralized training and decentralized execution framework is adopted to complete the training of hunting strategy. Each USV in the fleet shares the same policy and perform actions independently. Simulation experiments have verified the effectiveness of the DPOMH-PPO algorithm in the test scenarios with different numbers of USVs. Moreover, the advantages of the proposed model are comprehensively analyzed from the aspects of algorithm performance, migration effect in task scenarios and self-organization capability after being damaged, the potential deployment and application of DPOMH-PPO in the real environment is verified.

Deep Reinforcement Learning Based Multi-UUV Cooperative Control for Target Capturing

Secure and Cooperative Target Tracking Via AUV Swarm - A Reinforcement Learning Approach.

Underwater Multi-agent Cooperative Formation Hunting Based on Deep Reinforcement Learning

Multi-USV Dynamic Navigation and Target Capture: A Guided Multi-Agent Reinforcement Learning Approach

Multi-AUV Cooperative Underwater Multi-Target Tracking Based on Dynamic-Switching-enabled Multi-Agent Reinforcement Learning

Cooperative Multi-Target Hunting by Unmanned Surface Vehicles Based on Multi-Agent Reinforcement Learning

Multi-USV Deep Reinforcement Learning for Distributed Cooperative Target Tracking

Multi-UAV Cooperative Target Tracking Based on Swarm Intelligence

Multi-Agent Reinforcement Learning Aided Intelligent UAV Swarm for Target Tracking

A Multi-agent Deep Reinforcement Learning Method for UAVs Cooperative Pursuit Problem

Distributed Information Fusion Based Trajectory Tracking for USV and UAV Clusters Via Multi-Agent Deep Learning Approach

Reinforcement Learning-Based Cooperative Hunting for an Unmanned Surface Vehicle Swarm

Multi-Agent Reinforcement Learning for Unmanned Surface Vehicle Hunting Target

An Investigation on Multi-UAVs Cooperative Control Algorithm for Target Chasing

Improving multi-target cooperative tracking guidance for UAV swarms using multi-agent reinforcement learning

Scalable-MADDPG-Based Cooperative Target Invasion for a Multi-USV System

An Effective Strategy for Distributed Unmanned Underwater Vehicles to Encircle and Capture Intelligent Targets

Guidance and Control of Autonomous Surface Underwater Vehicles for Target Tracking in Ocean Environment by Deep Reinforcement Learning

Multi-UAV Autonomous Collaborative Target Tracking in Dynamic Environment Based on Multi-agent Reinforcement Learning

Deep Reinforcement Learning Based Multi-Auvs Cooperative Decision-Making for Attack–defense Confrontation Missions