Abstract:Reinforcement Learning has been shown to have a great potential for robotics. It demonstrated the capability to solve complex manipulation and locomotion tasks, even by learning end-to-end policies that operate directly on visual input, removing the need for custom perception systems. However, for practical robotics applications, its scarce sample efficiency, the need for huge amounts of resources, data, and computation time can be an insurmountable obstacle. One potential solution to this sample efficiency issue is the use of simulated environments. However, the discrepancy in visual and physical characteristics between reality and simulation, namely the sim-to-real gap, often significantly reduces the real-world performance of policies trained within a simulator. In this work we propose a sim-to-real technique that trains a Soft-Actor Critic agent together with a decoupled feature extractor and a latent-space dynamics model. The decoupled nature of the method allows to independently perform the sim-to-real transfer of feature extractor and control policy, and the presence of the dynamics model acts as a constraint on the latent representation when finetuning the feature extractor on real-world data. We show how this architecture can allow the transfer of a trained agent from simulation to reality without retraining or finetuning the control policy, but using real-world data only for adapting the feature extractor. By avoiding training the control policy in the real domain we overcome the need to apply Reinforcement Learning on real-world data, instead, we only focus on the unsupervised training of the feature extractor, considerably reducing real-world experience collection requirements. We evaluate the method on sim-to-sim and sim-to-real transfer of a policy for table-top robotic object pushing. We demonstrate how the method is capable of adapting to considerable variations in the task observations, such as changes in point-of-view, colors, and lighting, all while substantially reducing the training time with respect to policies trained directly in the real.

Learning Tactilemotor Policy for Robotic Cable Following Via Sim-to-Real Transfer.

Modular Deep Q Networks for Sim-to-real Transfer of Visuo-motor Policies

Skill Generalization of Tubular Object Manipulation with Tactile Sensing and Sim2Real Learning

Visual-Tactile Multimodality for Following Deformable Linear Objects Using Reinforcement Learning

Planar Robot Casting with Real2Sim2Real Self-Supervised Learning

Dexterous In-Hand Manipulation of Slender Cylindrical Objects through Deep Reinforcement Learning with Tactile Sensing

Multi-Stage Cable Routing through Hierarchical Imitation Learning

Sim2Real Manipulation on Unknown Objects with Tactile-based Reinforcement Learning

Sim-to-Real Transfer for Robotic Manipulation with Tactile Sensory

Sim-to-real via latent prediction: Transferring visual non-prehensile manipulation policies

Multistage Cable Routing Through Hierarchical Imitation Learning

General-Purpose Sim2Real Protocol for Learning Contact-Rich Manipulation With Marker-Based Visuotactile Sensors

Position Control of Cable-Driven Robotic Soft Arm Based on Deep Reinforcement Learning

Cable manipulation with a tactile-reactive gripper

One-shot sim-to-real transfer policy for robotic assembly via reinforcement learning with visual demonstration

Robot Manipulation Skills Transfer for Sim-to-Real in Unstructured Environments

Utilizing Reinforcement Learning to Drive Redundant Constrained Cable-Driven Robots with Unknown Parameters

Real2Sim2Real Transfer for Control of Cable-driven Robots via a Differentiable Physics Engine

Sim-to-Real Transfer of Soft Robotic Navigation Strategies That Learns from the Virtual Eye-in-Hand Vision

Curriculum-based Sensing Reduction in Simulation to Real-World Transfer for In-hand Manipulation

Sim-to-Real Transfer of Compliant Bipedal Locomotion on Torque Sensor-Less Gear-Driven Humanoid