Abstract:As an emerging direction of multi-agent collaborative control technology, multiple autonomous underwater vehicle (multi-AUV) cooperative area search technology has played an important role in civilian fields such as marine resource exploration and development, marine rescue, and marine scientific expeditions, as well as in military fields such as mine countermeasures and military underwater reconnaissance. At present, as we continue to explore the ocean, the environment in which AUVs perform search tasks is mostly unknown, with many uncertainties such as obstacles, which places high demands on the autonomous decision-making capabilities of AUVs. Moreover, considering the limited detection capability of a single AUV in underwater environments, while the area searched by the AUV is constantly expanding, a single AUV cannot obtain global state information in real time and can only make behavioral decisions based on local observation information, which adversely affects the coordination between AUVs and the search efficiency of multi-AUV systems. Therefore, in order to face increasingly challenging search tasks, we adopt multi-agent reinforcement learning (MARL) to study the problem of multi-AUV cooperative area search from the perspective of improving autonomous decision-making capabilities and collaboration between AUVs. First, we modeled the search task as a decentralized partial observation Markov decision process (Dec-POMDP) and established a search information map. Each AUV updates the information map based on sonar detection information and information fusion between AUVs, and makes real-time decisions based on this to better address the problem of insufficient observation information caused by the weak perception ability of AUVs in underwater environments. Secondly, we established a multi-AUV cooperative area search system (MACASS), which employs a search strategy based on multi-agent reinforcement learning. The system combines various AUVs into a unified entity using a distributed control approach. During the execution of search tasks, each AUV can make action decisions based on sonar detection information and information exchange among AUVs in the system, utilizing the MARL-based search strategy. As a result, AUVs possess enhanced autonomy in decision-making, enabling them to better handle challenges such as limited detection capabilities and insufficient observational information.

Adaptive Meta-Reinforcement Learning for AUVs 3D Guidance and Control under Unknown Ocean Currents

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

AUV Path Following Control using Deep Reinforcement Learning Under the Influence of Ocean Currents.

Continuous Control for Autonomous Underwater Vehicle Path Following Using Deep Interactive Reinforcement Learning

An Information-Assisted Deep Reinforcement Learning Path Planning Scheme for Dynamic and Unknown Underwater Environment

A Path Planning Method Based on Deep Reinforcement Learning for AUV in Complex Marine Environment

Simultaneous Control and Guidance of an AUV Based on Soft Actor-Critic

Adaptive Energy-Efficient Reinforcement Learning for AUV 3D Motion Planning in Complex Underwater Environments

Sim-to-Real Transfer of Adaptive Control Parameters for AUV Stabilization under Current Disturbance

Deep Reinforcement Learning with Model Predictive Control for Path Following of Autonomous Underwater Vehicle

Three-Dimensional Path-Following Control of an Autonomous Underwater Vehicle Based on Deep Reinforcement Learning

Reinforcement Learning Based Obstacle Avoidance for AUV Swarm in Dynamic Ocean Environment

Attention-Based Meta-Reinforcement Learning for Tracking Control of AUV with Time-Varying Dynamics

Target Search Control Of Auv In Underwater Environment With Deep Reinforcement Learning

Hybrid offline-online reinforcement learning for obstacle avoidance in autonomous underwater vehicles

Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle

End-To-End Sensorimotor Control Problems Of Auvs With Deep Reinforcement Learning

A General Motion Controller Based on Deep Reinforcement Learning for an Autonomous Underwater Vehicle with Unknown Disturbances.

A Method for Multi-AUV Cooperative Area Search in Unknown Environment Based on Reinforcement Learning

Research on obstacle avoidance of underactuated autonomous underwater vehicle based on offline reinforcement learning

AUV Obstacle Avoidance Framework Based on Event-Triggered Reinforcement Learning