Abstract:Unmanned surface vehicle (USV) is a robotic system with autonomous planning, driving, and navigation capabilities. With the continuous development of applications, the missions faced by USV are becoming more and more complex, so it is difficult for a single USV to meet the mission requirements. Compared with a single USV, a multi-USV system has some outstanding advantages such as fewer perceptual constraints, larger operation ranges, and stronger operation capability. In the search mission about multiple stationary underwater targets by a multi-USV system in the environment with obstacles, we propose a novel cooperative search algorithm (CSBDRL) based on reinforcement learning (RL) method and probability map method. CSBDRL is composed of the environmental sense module and policy module, which are organized by the “divide and conquer” policy-based architecture. The environmental sense module focuses on providing environmental sense values by using the probability map method. The policy module focuses on learning the optimal policy by using RL method. In CSBDRL, the mission environment is modeled and the corresponding reward function is designed to effectively explore the environment and learning policies. We test CSBDRL in the simulation environment and compare it with other methods. The results prove that compared with other methods, CSBDRL makes the multi-USV system have a higher search efficiency, which can ensure targets are found more quickly and accurately while ensuring the USV avoids obstacles in time during the mission.

Dynamic Hunting Method for Multi-USVs Based on Improved Game Theory Model

A Cooperative Hunting Method for Multi-USV Based on the A* Algorithm in an Environment with Obstacles

Multi-Agent Reinforcement Learning for Unmanned Surface Vehicle Hunting Target

Underwater Multi-agent Cooperative Formation Hunting Based on Deep Reinforcement Learning

Optimal Path Planning of Unmanned Surface Vehicle under Current Environment

A Hybrid Path Planning Algorithm for Unmanned Surface Vehicles in Complex Environment with Dynamic Obstacles.

A Cooperative Hunting Method for Multi-AUV Swarm in Underwater Weak Information Environment with Obstacles

Research on Real-Time Collision Avoidance and Path Planning of Usvs in Multi-Obstacle Ships Environment

Dynamic Area Coverage for Multi-USV with Fully Connected Constraint Based on Game Theory

Multi-USV System Cooperative Underwater Target Search Based on Reinforcement Learning and Probability Map

Path planning and search effectiveness of USV based on underwater target scattering model

Multiple targets traversing for unmanned surface vehicles by bundled genetic optimization and fast-marching Q-Learning

A dynamic velocity potential field method for multi-AUV cooperative hunting tasks

Model-Driven Cooperative Path Planning for Dynamic Target Searching of Unmanned Unterwater Vehicle Formation

Path planning for Multi-USV target coverage in complex environments

Pursuit-Evasion Game of Unmanded Surface Vehicles Based on Deep Reinforcement Learning

Collision Avoidance Strategies for Unmanned Surface Vehicles Based on Probabilistic Game Theory Framework

Autonomous Obstacle Avoidance Algorithm for Unmanned Surface Vehicles Based on an Improved Velocity Obstacle Method

Optimal search path planning for unmanned surface vehicle based on an improved genetic algorithm

3U: Joint Design of UAV-USV-UUV Networks for Cooperative Target Hunting

Adaptive USVs Swarm Optimization for Target Tracking in Dynamic Environments