Abstract:Reinforcement Learning has been shown to have a great potential for robotics. It demonstrated the capability to solve complex manipulation and locomotion tasks, even by learning end-to-end policies that operate directly on visual input, removing the need for custom perception systems. However, for practical robotics applications, its scarce sample efficiency, the need for huge amounts of resources, data, and computation time can be an insurmountable obstacle. One potential solution to this sample efficiency issue is the use of simulated environments. However, the discrepancy in visual and physical characteristics between reality and simulation, namely the sim-to-real gap, often significantly reduces the real-world performance of policies trained within a simulator. In this work we propose a sim-to-real technique that trains a Soft-Actor Critic agent together with a decoupled feature extractor and a latent-space dynamics model. The decoupled nature of the method allows to independently perform the sim-to-real transfer of feature extractor and control policy, and the presence of the dynamics model acts as a constraint on the latent representation when finetuning the feature extractor on real-world data. We show how this architecture can allow the transfer of a trained agent from simulation to reality without retraining or finetuning the control policy, but using real-world data only for adapting the feature extractor. By avoiding training the control policy in the real domain we overcome the need to apply Reinforcement Learning on real-world data, instead, we only focus on the unsupervised training of the feature extractor, considerably reducing real-world experience collection requirements. We evaluate the method on sim-to-sim and sim-to-real transfer of a policy for table-top robotic object pushing. We demonstrate how the method is capable of adapting to considerable variations in the task observations, such as changes in point-of-view, colors, and lighting, all while substantially reducing the training time with respect to policies trained directly in the real.

ROSO: Improving Robotic Policy Inference via Synthetic Observations

Scaling Robot Learning with Semantically Imagined Experience

RoSSO: A High-Performance Python Package for Robotic Surveillance Strategy Optimization Using JAX

Enabling Novel Mission Operations and Interactions with ROSA: The Robot Operating System Agent

RoVi-Aug: Robot and Viewpoint Augmentation for Cross-Embodiment Robot Learning

Autonomous Improvement of Instruction Following Skills via Foundation Models

Sim-to-real via latent prediction: Transferring visual non-prehensile manipulation policies

ROSA: Random Subspace Adaptation for Efficient Fine-Tuning

RoCoDA: Counterfactual Data Augmentation for Data-Efficient Robot Learning from Demonstrations

Automated Creation of Digital Cousins for Robust Policy Learning

Mirage: Cross-Embodiment Zero-Shot Policy Transfer with Cross-Painting

RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulation

Scaling Robot Policy Learning via Zero-Shot Labeling with Foundation Models

Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Models

Robot learning on the job: Human-in-the-loop autonomy and learning during deployment

IntervenGen: Interventional Data Generation for Robust and Data-Efficient Robot Imitation Learning

One-Step Diffusion Policy: Fast Visuomotor Policies via Diffusion Distillation

Few-Shot Image-to-Semantics Translation for Policy Transfer in Reinforcement Learning

P3-PO: Prescriptive Point Priors for Visuo-Spatial Generalization of Robot Policies

RISE: 3D Perception Makes Real-World Robot Imitation Simple and Effective

Synthetica: Large Scale Synthetic Data for Robot Perception