Abstract:The learning inefficiency of reinforcement learning (RL) from scratch hinders its practical application towards continuous robotic tracking control, especially for high-dimensional robots. This work proposes a data-informed residual reinforcement learning (DR-RL) based robotic tracking control scheme applicable to robots with high dimensionality. The proposed DR-RL methodology outperforms common RL methods regarding sample efficiency and scalability. Specifically, we first decouple the original robot into low-dimensional robotic subsystems; and further utilize one-step backward (OSBK) data to construct incremental subsystems that are equivalent model-free representations of the above decoupled robotic subsystems. The formulated incremental subsystems allow for parallel learning to relieve computation load and offer us mathematical descriptions of robotic movements for conducting theoretical analysis. Then, we apply DR-RL to learn the tracking control policy, a combination of incremental base policy and incremental residual policy, under a parallel learning architecture. The incremental residual policy uses the guidance from the incremental base policy as the learning initialization and further learns from interactions with environments to endow the tracking control policy with adaptability towards dynamically changing environments. Our proposed DR-RL based tracking control scheme is developed with rigorous theoretical analysis of system stability and weight convergence. The effectiveness of our proposed method is validated numerically on a 7-DoF KUKA iiwa robot manipulator and experimentally on a 3-DoF robot manipulator that would fail for other counterpart RL methods.

Reinforcement Learning Tracking Control for Robotic Manipulator With Kernel-Based Dynamic Model

Variable Force Tracking Control Via Online Learning

Unified Motion/Force/Impedance Control for Manipulators in Unknown Contact Environments Based on Robust Model-Reaching Approach

Offline Reinforcement Learning of Robotic Control Using Deep Kinematics and Dynamics

Data Informed Residual Reinforcement Learning for High-Dimensional Robotic Tracking Control

Fast and Accurate Tracking Control of Robotic Manipulators Subject to State Constraints and Input Saturation by Effectively Integrating Planning Strategies

Reinforcement Learning Based Control for Uncertain Robotic Manipulator Trajectory Tracking

Deep reinforcement learning with reward shaping for tracking control and vibration suppression of flexible link manipulator

Multi-Kernel Online Reinforcement Learning for Path Tracking Control of Intelligent Vehicles

Control Of Flexible Manipulator Based On Reinforcement Learning

A Reinforcement-Learning-Based Adaptive Sliding Mode Controller for Robotic Manipulators

Event-triggered Learning-Based Robust Tracking Control for Robotic Manipulators with Uncertain Dynamics and Non-Zero Equilibrium

Tracking Control for Robot Manipulator Based on Deterministic Learning and Event-Triggered Mechanism

Reinforcement learning-based adaptive tracking control for flexible-joint robotic manipulators

Optimal Tracking Control of Mechatronic Servo System Using Integral Reinforcement Learning

Model-free tracking control of complex dynamical trajectories with machine learning

Adaptive sliding mode control of robotic manipulator based on reinforcement learning

Model-Based Reinforcement Learning for Closed-Loop Dynamic Control of Soft Robotic Manipulators

Manipulator Trajectory Optimization Using Reinforcement Learning on a Reduced-Order Dynamic Model with Deep Neural Network Compensation

Model-Free Multi-Kernel Learning Control for Nonlinear Discrete-Time Systems.