Abstract:This paper presents a learning-based approach for centralized position control of Tendon Driven Continuum Robots (TDCRs) using Deep Reinforcement Learning (DRL), with a particular focus on the Sim-to-Real transfer of control policies. The proposed control method employs the Modified Transpose Jacobian (MTJ) control strategy, with its parameters optimally tuned using the Deep Deterministic Policy Gradient (DDPG) algorithm. Classical model-based controllers encounter significant challenges due to the inherent uncertainties and nonlinear dynamics of continuum robots. In contrast, model-free control strategies require efficient gain-tuning to handle diverse operational scenarios. This research aims to develop a model-free controller with performance comparable to model-based strategies by integrating an optimal adaptive gain-tuning system. Both simulations and real-world implementations demonstrate that the proposed method significantly enhances the trajectory-tracking performance of continuum robots independent of initial conditions and paths within the operational task-space, effectively establishing a task-free controller.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: How to achieve centralized position control of Tendon - Driven Continuum Robots (TDCRs) through Deep Reinforcement Learning (DRL), especially optimizing the control strategy in the process of Sim - to - Real transfer. Specifically, the paper focuses on the following points: 1. **Control Challenges**: - Traditional model - based controllers face significant challenges when dealing with the nonlinear dynamics and inherent uncertainties of continuum robots. - Model - based controllers are difficult to adapt to complex operating scenarios, while model - free control strategies require effective gain tuning to handle different tasks. 2. **Research Objectives**: - Develop a model - free controller whose performance can be comparable to model - based strategies, and achieve this by integrating an optimal adaptive gain tuning system. - Achieve effective transfer from simulation to reality, ensuring accurate trajectory tracking in uncertain and dynamic environments. 3. **Technical Methods**: - Propose a control strategy based on the Modified Transpose Jacobian (MTJ) and use the Deep Deterministic Policy Gradient (DDPG) algorithm to optimally tune its parameters. - Verify the proposed method through simulation and actual experiments, demonstrating a significant improvement in task - space trajectory tracking performance under different initial conditions and paths, thereby establishing a task - independent controller. 4. **Main Contributions**: - Achieve effective Sim - to - Real transfer, enabling the MTJ controller to achieve accurate trajectory tracking in uncertain and dynamic environments. - Use reinforcement learning as an optimal adaptive gain tuning system, reducing training time and improving the robustness and lightweight characteristics of the controller, making it suitable for practical applications. In summary, this paper aims to develop an efficient, robust, and practical model - free controller by combining deep reinforcement learning and the modified transpose Jacobian control strategy to solve the control problems of tendon - driven continuum robots in complex environments.

Learning-based Control for Tendon-Driven Continuum Robotic Arms

Open-Loop Motion Control of a Hydraulic Soft Robotic Arm Using Deep Reinforcement Learning

A Reinforcement Learning Approach for Continuum Robot Control

Integrating DeepRL with Robust Low-Level Control in Robotic Manipulators for Non-Repetitive Reaching Tasks

An Efficient Learning Control Framework With Sim-to-Real for String-Type Artificial Muscle-Driven Robotic Systems

Deep reinforcement learning with reward shaping for tracking control and vibration suppression of flexible link manipulator

RL + Model-based Control: Using On-demand Optimal Control to Learn Versatile Legged Locomotion

An Analytical Differential Kinematics-Based Method for Controlling Tendon-Driven Continuum Robots

Model-Based Reinforcement Learning for Position Control of Continuum Manipulators Actuated by Pneumatic Artificial Muscles

A Non-Linear Model Predictive Task-Space Controller Satisfying Shape Constraints for Tendon-Driven Continuum Robots

Efficient reinforcement learning control for continuum robots based on Inexplicit Prior Knowledge

DTC: Deep Tracking Control

Real-Robot Deep Reinforcement Learning: Improving Trajectory Tracking of Flexible-Joint Manipulator with Reference Correction

Coordinated Control Based on Reinforcement Learning for Dual-Arm Continuum Manipulators in Space Capture Missions

Data Informed Residual Reinforcement Learning for High-Dimensional Robotic Tracking Control

Deep Reinforcement Learning Control of Fully-Constrained Cable-Driven Parallel Robots

Learning Force Control for Contact-Rich Manipulation Tasks With Rigid Position-Controlled Robots

Axis-space framework for cable-driven soft continuum robot control via reinforcement learning

Offline Reinforcement Learning of Robotic Control Using Deep Kinematics and Dynamics

Combining Deep Reinforcement Learning with a Jerk-Bounded Trajectory Generator for Kinematically Constrained Motion Planning