Abstract:With the rapid development of deep learning, model-free reinforcement learning algorithms have achieved remarkable results in many fields. However, their high sample complexity and the potential for causing damage to environments and robots pose severe challenges for their application in real-world environments. Model-based reinforcement learning algorithms are often used to reduce the sample complexity. One limitation of these algorithms is the inevitable modeling errors. While the black-box model can fit complex state transition models, it ignores the existing knowledge of physics and robotics, especially studies of kinematic and dynamic models of the robotic manipulator. Compared with the black-box model, the physics-inspired deep models do not require specific knowledge of each system to obtain interpretable kinematic and dynamic models. In model-based reinforcement learning, these models can simulate the motion and be combined with classical controllers. This is due to their sharing the same form as traditional models, leading to higher precision tracking results. In this work, we utilize physics-inspired deep models to learn the kinematics and dynamics of a robotic manipulator. We propose a model-based offline reinforcement learning algorithm for controller parameter learning, combined with the traditional computed-torque controller. Experiments on trajectory tracking control of the Baxter manipulator, both in joint and operational space, are conducted in simulation and real environments. Experimental results demonstrate that our algorithm can significantly improve tracking accuracy and exhibits strong generalization and robustness.

Embodied Self-Supervised Learning (EMSSL) with Sampling and Training Coordination for Robot Arm Inverse Kinematic Model Learning

Embodied Self-Supervised Learning (EMSSL) with Sampling and Training Coordination for Robot Arm Inverse Kinematics Model Learning

CEMSSL: A Unified Framework for Multi-Solution Inverse Kinematic Model Learning of Robot Arms with High-Precision Manipulation

Learning Inverse Statics Models Efficiently

Foam-Embedded Soft Robotic Joint with Inverse Kinematic Modeling by Iterative Self-Improving Learning

An Integrated Physical-Learning Model of Physical Human-Robot Interactions with Application to Pose Estimation in Bikebot Riding

Robot Arm Configuration Self-Awareness Via Interacting with Environment

Full-Body Visual Self-Modeling of Robot Morphologies

Unsupervised Model Learning of Pneumatic Musculoskeletal Robots with Inadequate Sensors

Embodied Self-supervised Learning by Coordinated Sampling and Training

Learning-model-based Control for Robot Manipulators Sensorless Kinesthetic Teaching Using Sparse Feature Dynamics

Reconfigurable Robot Identification from Motion Data

Offline Reinforcement Learning of Robotic Control Using Deep Kinematics and Dynamics

Kinematic Modeling and Simulation of Dual-Arm Robot

Robust Kalman Filtering Cooperated Elman Neural Network Learning For Vision-Sensing-Based Robotic Manipulation With Global Stability

Self Model for Embodied Intelligence: Modeling Full-Body Human Musculoskeletal System and Locomotion Control with Hierarchical Low-Dimensional Representation

EMG-Based Continuous and Simultaneous Estimation of Arm Kinematics in Able-Bodied Individuals and Stroke Survivors

A Close-Loop EMG Model for Continuous Joint Movements Estimation of a Rehabilitation Robot

A Self-Organization Learning Algorithm for Visuo-Motor Coordination in Unstructured Environments.

Automating the Learning of Inverse Kinematics for Robotic Arms with Redundant DoFs

A Learning-Based Multiscale Modelling Approach to Real-Time Serial Manipulator Kinematics Simulation.