Abstract:Robot control tasks are typically solved by reinforcement learning approaches in a circular way of trial and learn. A recent trend of the research on robotic reinforcement learning is the employment of the deep learning methods. Existing deep learning methods achieve the control by training the approximation models of the dynamic function, value function or the policy function in the control algorithms. However, these methods usually handle the modeling from a statistical perspective without considering the physics characteristics of the robot’s motion. One of the typical problems is the force transmission through different parts of a robot and the calculations of the robotic dynamics quantities are prone to be ignored. To this end, we propose to use the force transmission graph to interpret the force transmission mechanism obeyed by the motion of the robot and estimate the dynamics quantities of the robot’s motion with a quadratic model. Following this concern, we propose a model-based reinforcement learning framework for robotic control in which the dynamic model comprises two components, i.e. the Graph Convolution Network (GCN) and the Two-Layer Perception (TLP) network. The GCN serves as a parameter estimator of the force transmission graph and a structural feature extractor. The TLP network approximates the quadratic model that should be able to estimate the dynamics quantities of the robot’s motion. For this reason, the proposed framework is named as GCN of Dynamics estimation in Reinforcement Learning method (GDRL for short). The deployed method interprets the intrinsic mechanism of robotic force transmissions through robot limbs, therefore the model is highly interpretable. Experimental results show that GDRL can predict the gesture and location of the robot for the next move well such that the performance of our method surpasses that of the previous methods in robot control task in our task setting of multiple types of robots. We also designed to compare with the previous model-free methods in our task setting, and the results are outstanding, which is owing to the interpretation of the physical characteristics.

Graph-geometric message passing via a graph convolution transformer for FKP regression

DisGNet: A Distance Graph Neural Network for Forward Kinematics Learning of Gough-Stewart Platform

Dynamic Modeling and Robust Control in Task Space of 4-DOF Parallel Electric Platform with 4TPS-1PS Structure

A Cascaded Deep Learning Framework for Real-time and Robust Grasp Planning

Graph and dynamics interpretation in robotic reinforcement learning task

Euclidean Equivariant Models for Generative Graphical Inverse Kinematics

JGR-P2O: Joint Graph Reasoning Based Pixel-to-Offset Prediction Network for 3D Hand Pose Estimation from a Single Depth Image

Dynamic 3D Gaussian Tracking for Graph-Based Neural Dynamics Modeling

Online 6DoF Pose Estimation in Forests using Cross-View Factor Graph Optimisation and Deep Learned Re-localisation

Deep Kinematic Pose Regression.

Learning Deep Nets for Gravitational Dynamics with Unknown Disturbance through Physical Knowledge Distillation: Initial Feasibility Study

Comparison of Graph Fitting and Sparse Deep Learning Model for Robot Pose Estimation

Graph-Transporter: A Graph-based Learning Method for Goal-Conditioned Deformable Object Rearranging Task

Drone Detection and Pose Estimation Using Relational Graph Networks

GPHQP: Hierarchical Quadratic Programming for Controlling a Gaussian Process Regression Model of Redundant Robot

Learning Dynamical Human-Joint Affinity for 3D Pose Estimation in Videos

SPGformer: Serial–Parallel Hybrid GCN-Transformer With Graph-Oriented Encoder for 2-D-to-3-D Human Pose Estimation

Exploring Multiple Geometric Representations for 6DoF Object Pose Estimation

GaPT: Gaussian Process Toolkit for Online Regression with Application to Learning Quadrotor Dynamics

Multiscale Spatio-Temporal Graph Neural Networks for 3D Skeleton-Based Motion Prediction

Joint estimation of pose, depth, and optical flow with a competition-cooperation transformer network