Abstract:Autonomous manipulation operations represent the high intelligent coordination from robotic vision and control, it is also a symbol of the advances of robotic intelligence. The limitations of visual sensing and the increasingly complex experimental conditions make autonomous manipulation operations more difficult, particularly for deep reinforcement learning methods, which can enhance robotic control intelligence but require a lot of training process. Due to the high-dimensional continuous state space and continuous action space characteristics of underwater operations, this paper adopts a policy-based reinforcement learning method as the foundational approach. To address the issues of instability and low convergence efficiency in traditional policy-based reinforcement learning algorithms during the learning process, this paper proposes a novel policy learning method. This method adopts the Proximal Policy Optimization algorithm (PPOClip) and optimizes it through an actor-critic network. The aim is to improve the stability and effectiveness of convergence in the learning process. In the underwater training environment, a new reward shaping scheme has been designed to address the issue of reward sparsity during the training process. The manually crafted dense reward function is utilized as attractive and repulsive potential functions for goal manipulation and obstacle avoidance. On the highly complex underwater manipulation and training environment, transferred learning algorithm has been established to reduce the training times and compensate the differences between the simulation and experiment. Simulations and tank experiments have verified the performance of the proposed strategy learning method.

A Deep Reinforcement Learning-based Decentralized Hierarchical Motion Control Strategy for Multiple Amphibious Spherical Robot Systems with Tilting Thrusters

Three-Dimensional Coordination Control for Multiple Autonomous Underwater Vehicles

AUV position tracking and trajectory control based on fast-deployed deep reinforcement learning method

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

Collaboration and Task Planning of Turtle-Inspired Multiple Amphibious Spherical Robots

Deep reinforcement learning for adaptive path planning and control of an autonomous underwater vehicle

Underwater Multi-agent Cooperative Formation Hunting Based on Deep Reinforcement Learning

Open-Loop Motion Control of a Hydraulic Soft Robotic Arm Using Deep Reinforcement Learning

Research and Design of an Autonomous Underwater Vehicle Path Planning Method Based on Deep Reinforcement Learning

Adaptive formation motion planning and control of autonomous underwater vehicles using deep reinforcement learning

Reinforcement learning control method for real‐time hybrid simulation based on deep deterministic policy gradient algorithm

Dynamic Analysis and Path Planning of a Turtle-Inspired Amphibious Spherical Robot

AUV Path Following Control using Deep Reinforcement Learning Under the Influence of Ocean Currents.

Learning strategies for underwater robot autonomous manipulation control

Single degree of freedom control based on deep reinforcement learning for underwater unmanned vehicle

Path-Following Control of Unmanned Underwater Vehicle Based on an Improved TD3 Deep Reinforcement Learning

Residual Reinforcement Learning for Motion Control of a Bionic Exploration Robot—RoboDact

Learning to Swim: Reinforcement Learning for 6-DOF Control of Thruster-driven Autonomous Underwater Vehicles

Study on the Autonomous Multirobot Collaborative Control System Based on Spherical Amphibious Robots

Control and Coordination of a SWARM of Unmanned Surface Vehicles using Deep Reinforcement Learning in ROS