Abstract:Generalization remains one of the most important desiderata for robust robot learning systems. While recently proposed approaches show promise in generalization to novel objects, semantic concepts, or visual distribution shifts, generalization to new tasks remains challenging. For example, a language-conditioned policy trained on pick-and-place tasks will not be able to generalize to a folding task, even if the arm trajectory of folding is similar to pick-and-place. Our key insight is that this kind of generalization becomes feasible if we represent the task through rough trajectory sketches. We propose a policy conditioning method using such rough trajectory sketches, which we call RT-Trajectory, that is practical, easy to specify, and allows the policy to effectively perform new tasks that would otherwise be challenging to perform. We find that trajectory sketches strike a balance between being detailed enough to express low-level motion-centric guidance while being coarse enough to allow the learned policy to interpret the trajectory sketch in the context of situational visual observations. In addition, we show how trajectory sketches can provide a useful interface to communicate with robotic policies: they can be specified through simple human inputs like drawings or videos, or through automated methods such as modern image-generating or waypoint-generating methods. We evaluate RT-Trajectory at scale on a variety of real-world robotic tasks, and find that RT-Trajectory is able to perform a wider range of tasks compared to language-conditioned and goal-conditioned policies, when provided the same training data.

What problem does this paper attempt to address?

This paper aims to address the task generalization problem in robotic learning systems, particularly the challenges encountered when facing new tasks. Specifically, even though language-conditioned policies perform well on known tasks such as pick-and-place, they fail to directly generalize to new tasks such as folding, despite the similarities in arm trajectories. To overcome this limitation, the researchers propose a method called "Hindsight Trajectory Sketches" (RT-Trajectory). The core of RT-Trajectory lies in using rough trajectory sketches as a way to condition policies. These sketches are detailed enough to convey low-level motion guidance, yet rough enough to allow the learned policies to interpret the trajectories based on current visual observations. This approach balances expressiveness and usability, enabling policies to effectively execute new tasks that would otherwise be challenging to handle. Additionally, trajectory sketches provide a useful interface for communication with robot policies, which can be specified through simple hand-drawn sketches, videos, or modern image generation methods. The effectiveness of RT-Trajectory is evaluated through a series of real-world robotic tasks in the paper. The results demonstrate that, compared to language-conditioned and goal-conditioned policies, RT-Trajectory can perform a wider range of tasks given the same training data, especially excelling in generalization to new actions. The experiments also showcase the ability of RT-Trajectory to generalize to new skills by combining different observed states and actions, as well as completely unseen states or actions. The researchers validate the advantages of RT-Trajectory over other baseline methods through comparative experiments and discuss its generalization capability and robustness in facing various distribution changes in real-world scenarios.

RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory Sketches

Demonstration Data-Driven Parameter Adjustment for Trajectory Planning in Highly Constrained Environments

Generalize Robot Learning from Demonstration to Variant Scenarios with Evolutionary Policy Gradient

Domain Generalization for Vision-based Driving Trajectory Generation

Learning Generalizable Tool-use Skills through Trajectory Generation

Any-point Trajectory Modeling for Policy Learning

RT-Sketch: Goal-Conditioned Imitation Learning from Hand-Drawn Sketches

RT-Affordance: Affordances are Versatile Intermediate Representations for Robot Manipulation

Trajectory Optimization for Manipulation Considering Grasp Selection and Adjustment

Human skill knowledge guided global trajectory policy reinforcement learning method

Generalization of Task Parameterized Dynamical Systems using Gaussian Process Transportation

Adversarially Regularized Policy Learning Guided by Trajectory Optimization

Model Based Reinforcement Learning for Robot Grasping Trajectory Generation.

Robot Trajectron: Trajectory Prediction-based Shared Control for Robot Manipulation

Reparameterized Policy Learning for Multimodal Trajectory Optimization

Trajectory Advancement for Robot Stand-up with Human Assistance

An Efficient Generalizable Framework for Visuomotor Policies via Control-aware Augmentation and Privilege-guided Distillation

Diversity-based Trajectory and Goal Selection with Hindsight Experience Replay

Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisition

Trajectory-Oriented Policy Optimization with Sparse Rewards

Trajectory Optimization for Unknown Constrained Systems using Reinforcement Learning