Abstract:Autonomous manipulation operations represent the high intelligent coordination from robotic vision and control, it is also a symbol of the advances of robotic intelligence. The limitations of visual sensing and the increasingly complex experimental conditions make autonomous manipulation operations more difficult, particularly for deep reinforcement learning methods, which can enhance robotic control intelligence but require a lot of training process. Due to the high-dimensional continuous state space and continuous action space characteristics of underwater operations, this paper adopts a policy-based reinforcement learning method as the foundational approach. To address the issues of instability and low convergence efficiency in traditional policy-based reinforcement learning algorithms during the learning process, this paper proposes a novel policy learning method. This method adopts the Proximal Policy Optimization algorithm (PPOClip) and optimizes it through an actor-critic network. The aim is to improve the stability and effectiveness of convergence in the learning process. In the underwater training environment, a new reward shaping scheme has been designed to address the issue of reward sparsity during the training process. The manually crafted dense reward function is utilized as attractive and repulsive potential functions for goal manipulation and obstacle avoidance. On the highly complex underwater manipulation and training environment, transferred learning algorithm has been established to reduce the training times and compensate the differences between the simulation and experiment. Simulations and tank experiments have verified the performance of the proposed strategy learning method.

Learning an End-To-End Policy for AUV Control Within Just Forty Minutes Using Parallel Simulation

Continuous Control for Autonomous Underwater Vehicle Path Following Using Deep Interactive Reinforcement Learning

Deep Reinforcement Learning Approach with Multiple Experience Pools for UAV's Autonomous Motion Planning in Complex Unknown Environments

MarineGym: Accelerated Training for Underwater Vehicles with High-Fidelity RL Simulation

A Fast Adaptive AUV Control Policy Based on Progressive Networks with Context Information

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

Learning strategies for underwater robot autonomous manipulation control

AUV position tracking and trajectory control based on fast-deployed deep reinforcement learning method

Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning

Research on Motion Attitude Control of Under-actuated Autonomous Underwater Vehicle Based on Deep Reinforcement Learning

An Improved Method towards Multi-UAV Autonomous Navigation Using Deep Reinforcement Learning

Imitation Learning from Imperfect Demonstrations for AUV Path Tracking and Obstacle Avoidance

Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle

Action Guidance-Based Deep Interactive Reinforcement Learning for AUV Path Planning

AUV Collision Avoidance Planning Method Based on Deep Deterministic Policy Gradient

Deep Reinforcement Learning for Continuous Docking Control of Autonomous Underwater Vehicles: A Benchmarking Study

Motion control of autonomous underwater vehicle based on physics-informed offline reinforcement learning

Asynchronous Multithreading Reinforcement-Learning-Based Path Planning and Tracking for Unmanned Underwater Vehicle

Target Search Control Of Auv In Underwater Environment With Deep Reinforcement Learning

End-to-end deep reinforcement learning for control of an autonomous underwater robot with an undulating propulsor

Intelligent AUV Surfacing Control in Network Attack Scenario