Abstract:As one of the commonly used vehicles for underwater detection, underwater robots are facing a series of problems. The real underwater environment is large-scale, complex, real-time and dynamic, and many unknown obstacles may exist in the underwater environment. Under such complex conditions and lack of prior knowledge, the existing path planning methods are difficult to plan, therefore they cannot effectively meet the actual demands. In response to these problems, a three-dimensional marine environment including multiple obstacles is established with the real ocean current data in this paper, which is consistent with the actual application scenarios. Then, we propose an N-step Priority Double DQN (NPDDQN) path planning algorithm, which potently realizes obstacle avoidance in the complex environment. In addition, this study proposes an experience screening mechanism, which screens the explored positive experience and improves its reuse rate, thus efficiently improving the algorithm stability in the dynamic environment. This paper verifies the better performance of reinforcement learning compared with a variety of traditional methods in three-dimensional underwater path planning. Underwater robots based on the proposed method have good autonomy and stability, which provides a new method for path planning of underwater robots. Note to Practitioners—The goal of this study is to provide a new solution for obstacle avoidance in path planning of underwater robots, which is consistent with the dynamic and real-time demands of the real environment. Existing underwater path planning researches lack a consistent environment with the actual application, and therefore we firstly construct a three-dimensional ocean environment with real ocean current data to provide support for the algorithms. Additionally, most of the algorithms are pre-planning methods or require long-time calculation, and there is little research on obstacle avoidance. In the face of obstacle changes, underwater robots with poor adaptability will cause performance decline and even economic losses. The proposed algorithm learns through interaction with the environment, and therefore it does not require any prior experience, and has good adaptability as well as fast inference speed. Especially, in the dynamic environment, algorithm performance is difficult to guarantee due to less positive experience in exploration. The proposed experience screening mechanism improves the stability of the algorithm, so that the underwater robot maintains stable performance in different dynamic environments.

Intelligent Control Strategy for Robotic Manta via CPG and Deep Reinforcement Learning

Locating, Trajectory Planning and Control of an Underwater Propeller Cleaning Manipulator

Novel Online Optimized Control for Underwater Pipe-Cleaning Robots

Design of a MNSM-based Controller for the Swimming Motion of a Snake-Like Robot

Learning Agile Swimming: An End-to-End Approach without CPGs

A Deep Reinforcement Learning-based Decentralized Hierarchical Motion Control Strategy for Multiple Amphibious Spherical Robot Systems with Tilting Thrusters

Learning strategies for underwater robot autonomous manipulation control

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

Path-following Control of Fish-like Robots: A Deep Reinforcement Learning Approach

Single degree of freedom control based on deep reinforcement learning for underwater unmanned vehicle

A reinforcement learning control approach for underwater manipulation under position and torque constraints

Residual Reinforcement Learning for Motion Control of a Bionic Exploration Robot—RoboDact

Control and Coordination of a SWARM of Unmanned Surface Vehicles using Deep Reinforcement Learning in ROS

Intelligent Path Planning of Underwater Robot Based on Reinforcement Learning

Research and Design of an Autonomous Underwater Vehicle Path Planning Method Based on Deep Reinforcement Learning

On Study of Intelligent Control Methods for a Bionic Dolphin Based on Sensory Feedback CPG Model

Target Tracking Control of a Biomimetic Underwater Vehicle Through Deep Reinforcement Learning

Enhancing efficiency and propulsion in bio-mimetic robotic fish through end-to-end deep reinforcement learning

Development and CPG-based control of a biomimetic robotic fish with advanced underwater mobility

Reinforcement Learning Based Obstacle Avoidance for AUV Swarm in Dynamic Ocean Environment

Neural Controller for Swimming Modes and Gait Transition on an Ostraciiform Fish Robot