RoboMorph: In-Context Meta-Learning for Robot Dynamics Modeling

Manuel Bianchi Bazzi,Asad Ali Shahid,Christopher Agia,John Alora,Marco Forgione,Dario Piga,Francesco Braghin,Marco Pavone,Loris Roveda

2024-09-18

Abstract:The landscape of Deep Learning has experienced a major shift with the pervasive adoption of Transformer-based architectures, particularly in Natural Language Processing (NLP). Novel avenues for physical applications, such as solving Partial Differential Equations and Image Vision, have been explored. However, in challenging domains like robotics, where high non-linearity poses significant challenges, Transformer-based applications are scarce. While Transformers have been used to provide robots with knowledge about high-level tasks, few efforts have been made to perform system identification. This paper proposes a novel methodology to learn a meta-dynamical model of a high-dimensional physical system, such as the Franka robotic arm, using a Transformer-based architecture without prior knowledge of the system's physical parameters. The objective is to predict quantities of interest (end-effector pose and joint positions) given the torque signals for each joint. This prediction can be useful as a component for Deep Model Predictive Control frameworks in robotics. The meta-model establishes the correlation between torques and positions and predicts the output for the complete trajectory. This work provides empirical evidence of the efficacy of the in-context learning paradigm, suggesting future improvements in learning the dynamics of robotic systems without explicit knowledge of physical parameters. Code, videos, and supplementary materials can be found at project website. See <a class="link-external link-https" href="https://sites.google.com/view/robomorph/" rel="external noopener nofollow">this https URL</a>

Robotics

What problem does this paper attempt to address?

The paper aims to address the problem of robotic dynamics modeling. Specifically, it proposes a new method that utilizes a Transformer-based architecture to learn meta-dynamics models of high-dimensional physical systems (e.g., the Franka robotic arm) without requiring prior knowledge of the system's physical parameters. The main objective is to predict quantities of interest (such as the end-effector's pose and joint positions) given the torque signals for each joint. This prediction is of significant value for integration into a robot's deep model predictive control framework. Additionally, the study trained a meta-model using a dataset generated through large-scale parallel simulations, demonstrating the model's effectiveness under different control inputs and providing empirical evidence that it is possible to effectively learn the dynamics of a robot without explicit knowledge of physical parameters. This work provides a theoretical foundation and technical support for future improvements in learning the dynamics of robotic systems.

RoboMorph: In-Context Meta-Learning for Robot Dynamics Modeling

Towards Modeling and Influencing the Dynamics of Human Learning

Bidirectional recurrent learning of inverse dynamic models for robots with elastic joints: a real-time real-world implementation

Kinematics Transformer: Solving The Inverse Modeling Problem of Soft Robots using Transformers

Body Transformer: Leveraging Robot Embodiment for Policy Learning

A Novel Meta Control Framework for Robot Arm Reaching with Changeable Configuration

From system models to class models: An in-context learning paradigm

Learning Soft Robot Dynamics using Differentiable Kalman Filters and Spatio-Temporal Embeddings

Offline Reinforcement Learning of Robotic Control Using Deep Kinematics and Dynamics

Reconfigurable Robot Identification from Motion Data

Online Decision MetaMorphFormer: A Casual Transformer-Based Reinforcement Learning Framework of Universal Embodied Intelligence

Model-Based Reinforcement Learning Variable Impedance Control for Human-Robot Collaboration

Enhanced Transformer architecture for in-context learning of dynamical systems

Dream to Manipulate: Compositional World Models Empowering Robot Imitation Learning with Imagination

VidMan: Exploiting Implicit Dynamics from Video Diffusion Model for Effective Robot Manipulation

RT-1: Robotics Transformer for Real-World Control at Scale

Multi-embodiment Legged Robot Control As a Sequence Modeling Problem

Learning to Design and Use Tools for Robotic Manipulation

Visuo-dynamic self-modelling of soft robotic systems

Adaptive Asynchronous Control Using Meta-learned Neural Ordinary Differential Equations

Extrapolation of Physics-Inspired Deep Networks in Learning Robot Inverse Dynamics