Abstract:To solve the problem of multi-target hunting by an unmanned surface vehicle (USV) fleet, a hunting algorithm based on multi-agent reinforcement learning is proposed. Firstly, the hunting environment and kinematic model without boundary constraints are built, and the criteria for successful target capture are given. Then, the cooperative hunting problem of a USV fleet is modeled as a decentralized partially observable Markov decision process (Dec-POMDP), and a distributed partially observable multi-target hunting Proximal Policy Optimization (DPOMH-PPO) algorithm applicable to USVs is proposed. In addition, an observation model, a reward function and the action space applicable to multi-target hunting tasks are designed. To deal with the dynamic change of observational feature dimension input by partially observable systems, a feature embedding block is proposed. By combining the two feature compression methods of column-wise max pooling (CMP) and column-wise average-pooling (CAP), observational feature encoding is established. Finally, the centralized training and decentralized execution framework is adopted to complete the training of hunting strategy. Each USV in the fleet shares the same policy and perform actions independently. Simulation experiments have verified the effectiveness of the DPOMH-PPO algorithm in the test scenarios with different numbers of USVs. Moreover, the advantages of the proposed model are comprehensively analyzed from the aspects of algorithm performance, migration effect in task scenarios and self-organization capability after being damaged, the potential deployment and application of DPOMH-PPO in the real environment is verified.

Dynamic Target Assignment by Unmanned Surface Vehicles Based on Reinforcement Learning

Dynamic Target Assignment Method for UCAV Fleet Using Memory Enhanced Dynamic UMDA

Underwater Multi-agent Cooperative Formation Hunting Based on Deep Reinforcement Learning

Dynamic Navigation and Area Assignment of Multiple USVs Based on Multi-Agent Deep Reinforcement Learning

Multi-Agent Reinforcement Learning for Unmanned Surface Vehicle Hunting Target

Multi-USV Dynamic Navigation and Target Capture: A Guided Multi-Agent Reinforcement Learning Approach

Multi-UAV Cooperative Target Assignment Method Based on Reinforcement Learning

Unmanned Surface Vessel Motion Intention Recognition Based on Deep Reinforcement Learning

A Multi-AUV Maritime Target Search Method for Moving and Invisible Objects Based on Multi-Agent Deep Reinforcement Learning

Unmanned Surface Vehicle Aided Maritime Data Collection Using Deep Reinforcement Learning

Multiple targets traversing for unmanned surface vehicles by bundled genetic optimization and fast-marching Q-Learning

Dynamic Task Assignment for Multi-AUV Cooperative Hunting

Maximizing UAV Coverage in Maritime Wireless Networks: A Multiagent Reinforcement Learning Approach

Cooperative Multi-Target Hunting by Unmanned Surface Vehicles Based on Multi-Agent Reinforcement Learning

Deep Reinforcement Learning-based Multi-AUV Task Allocation Algorithm in Underwater Wireless Sensor Networks

Dynamic Task Assignment and Path Planning for Multi-AUV System in Variable Ocean Current Environment

Differential Game-Based Deep Reinforcement Learning in Underwater Target Hunting Task

Multi-Ship Dynamic Weapon-Target Assignment Via Cooperative Distributional Reinforcement Learning with Dynamic Reward

Asynchronous Multithreading Reinforcement Control Decision Method for Unmanned Surface Vessel

End-to-End Trajectory Tracking Algorithm for Unmanned Surface Vehicle Using Reinforcement Learning

Path Planning Algorithm for Unmanned Surface Vessel Based on Multiobjective Reinforcement Learning