Abstract:Robot control tasks are typically solved by reinforcement learning approaches in a circular way of trial and learn. A recent trend of the research on robotic reinforcement learning is the employment of the deep learning methods. Existing deep learning methods achieve the control by training the approximation models of the dynamic function, value function or the policy function in the control algorithms. However, these methods usually handle the modeling from a statistical perspective without considering the physics characteristics of the robot’s motion. One of the typical problems is the force transmission through different parts of a robot and the calculations of the robotic dynamics quantities are prone to be ignored. To this end, we propose to use the force transmission graph to interpret the force transmission mechanism obeyed by the motion of the robot and estimate the dynamics quantities of the robot’s motion with a quadratic model. Following this concern, we propose a model-based reinforcement learning framework for robotic control in which the dynamic model comprises two components, i.e. the Graph Convolution Network (GCN) and the Two-Layer Perception (TLP) network. The GCN serves as a parameter estimator of the force transmission graph and a structural feature extractor. The TLP network approximates the quadratic model that should be able to estimate the dynamics quantities of the robot’s motion. For this reason, the proposed framework is named as GCN of Dynamics estimation in Reinforcement Learning method (GDRL for short). The deployed method interprets the intrinsic mechanism of robotic force transmissions through robot limbs, therefore the model is highly interpretable. Experimental results show that GDRL can predict the gesture and location of the robot for the next move well such that the performance of our method surpasses that of the previous methods in robot control task in our task setting of multiple types of robots. We also designed to compare with the previous model-free methods in our task setting, and the results are outstanding, which is owing to the interpretation of the physical characteristics.

Dynamic Inference on Graphs using Structured Transition Models

Graph and dynamics interpretation in robotic reinforcement learning task

Newton-Cotes Graph Neural Networks: on the Time Evolution of Dynamic Systems

SENSORIMOTOR GRAPH: Action-Conditioned Graph Neural Network for Learning Robotic Soft Hand Dynamics

Learning Differentiable Tensegrity Dynamics using Graph Neural Networks

Modeling social interaction dynamics using temporal graph networks

Graph Neural Networks for Decentralized Multi-Robot Submodular Action Selection

Dynamic 3D Gaussian Tracking for Graph-Based Neural Dynamics Modeling

Learning to Dynamically Coordinate Multi-Robot Teams in Graph Attention Networks

Model-Based Control with Sparse Neural Dynamics

KG-Planner: Knowledge-Informed Graph Neural Planning for Collaborative Manipulators

Learning Scalable Decentralized Controllers for Heterogeneous Robot Swarms with Graph Neural Networks

Continuous-Time Dynamic Graph Learning Via Neural Interaction Processes.

Neural-Learning Trajectory Tracking Control of Flexible-Joint Robot Manipulators with Unknown Dynamics

Dynamic Motion Planning Model for Multirobot Using Graph Neural Network and Historical Information

Applying vision-guided graph neural networks for adaptive task planning in dynamic human robot collaborative scenarios

Active Training Trajectory Generation for Inverse Dynamics Model Learning with Deep Neural Networks

Dynamic-Resolution Model Learning for Object Pile Manipulation

Learning Dynamics and Structure of Complex Systems Using Graph Neural Networks

Neural Dynamics on Complex Networks

Learning-based Motion Planning in Dynamic Environments Using GNNs and Temporal Encoding