Abstract:Sim-to-Real refers to the process of transferring policies learned in simulation to the real world, which is crucial for achieving practical robotics applications. However, recent Sim2real methods either rely on a large amount of augmented data or large learning models, which is inefficient for specific tasks. In recent years, radiance field-based reconstruction methods, especially the emergence of 3D Gaussian Splatting, making it possible to reproduce realistic real-world scenarios. To this end, we propose a novel real-to-sim-to-real reinforcement learning framework, RL-GSBridge, which introduces a mesh-based 3D Gaussian Splatting method to realize zero-shot sim-to-real transfer for vision-based deep reinforcement learning. We improve the mesh-based 3D GS modeling method by using soft binding constraints, enhancing the rendering quality of mesh models. We then employ a GS editing approach to synchronize rendering with the physics simulator, reflecting the interactions of the physical robot more accurately. Through a series of sim-to-real robotic arm experiments, including grasping and pick-and-place tasks, we demonstrate that RL-GSBridge maintains a satisfactory success rate in real-world task completion during sim-to-real transfer. Furthermore, a series of rendering metrics and visualization results indicate that our proposed mesh-based 3D Gaussian reduces artifacts in unstructured objects, demonstrating more realistic rendering performance.

What problem does this paper attempt to address?

### Problems Addressed by the Paper This paper aims to address the Sim-to-Real transfer problem in robotic manipulation tasks. Specifically, existing Sim-to-Real methods either rely on a large amount of augmented data or large learning models, which are inefficient for specific tasks. To overcome these issues, the authors propose a new reinforcement learning framework based on 3D Gaussian Splatting—RL-GSBridge, to achieve zero-sample Sim-to-Real transfer. ### Solution 1. **Real2Sim Conversion**: - Use consumer-grade cameras to capture real-world image data and construct geometrically and texturally accurate simulation models through a 3D Gaussian Splatting method with soft mesh binding. - Propose a soft mesh binding method that improves rendering quality and modeling capability by relaxing the constraints of hard mesh binding, especially when dealing with non-rigid objects. 2. **Sim2Real Conversion**: - Use a physics-driven 3D Gaussian Splatting editing method in the simulation environment to ensure consistency between visual representation and physical interaction processes. - Train the robotic arm's manipulation strategy through Deep Reinforcement Learning (DRL) and directly apply the learned strategy to the real world without further training or fine-tuning. ### Main Contributions 1. **Novel Sim2Real RL Framework**: - Utilize the high-fidelity rendering of 3D Gaussian Splatting and the convenience of consumer-grade cameras to build realistic simulation environments. 2. **3D Gaussian Splatting Modeling Method with Soft Mesh Binding**: - Propose a soft mesh binding strategy to replace the hard mesh binding baseline, enhancing flexibility and rendering quality. 3. **Physics-Driven 3D Gaussian Splatting Editing**: - Synchronize dynamic change information provided by the simulator with the 3D Gaussian Splatting model to reflect real physical robot interactions. 4. **Validation on Real Physical Robots**: - Test the RL-GSBridge framework in real-world scenarios through grasping and placing tasks, demonstrating its efficient performance with complex textures and geometries. ### Experimental Results - **Grasping Task**: - In various complex manipulation scenarios, RL-GSBridge showed an average success rate drop of 6.6% in Sim-to-Real transfer, significantly better than the 80% success rate drop of traditional methods (e.g., RL-sim). - Particularly when handling unstructured objects (e.g., teddy bears), RL-GSBridge not only avoided a drop in success rate but also improved the success rate in some cases. - **Placing Task**: - In the cake placing task, RL-GSBridge improved the success rate in real environments by 4.54%, mainly because the real environment can tolerate some minor contacts that do not affect the overall task. - **Behavior Consistency Comparison**: - In both simulation and real scenarios, RL-GSBridge exhibited highly consistent behavior, including first-person perspective perception and robotic arm posture, whereas traditional RL-sim methods showed significant differences. ### Conclusion By applying recent successful radiance field reconstruction methods, RL-GSBridge demonstrated a high Sim-to-Real success rate in desktop-level robotic manipulation tasks. Future research directions include applying and improving this method on more diverse platforms and developing adaptive learning capabilities.

RL-GSBridge: 3D Gaussian Splatting Based Real2Sim2Real Method for Robotic Manipulation Learning

RoboGSim: A Real2Sim2Real Robotic Gaussian Splatting Simulator

SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splatting

Robo-GS: A Physics Consistent Spatial-Temporal Model for Robotic Arm with Hybrid Representation

Close the Sim2real Gap via Physically-based Structured Light Synthetic Data Simulation

Real2Sim or Sim2Real: Robotics Visual Insertion using Deep Reinforcement Learning and Real2Sim Policy Adaptation

Part-Guided 3D RL for Sim2Real Articulated Object Manipulation

Real–Sim–Real Transfer for Real-World Robot Control Policy Learning with Deep Reinforcement Learning

Reinforcement Learning with Generalizable Gaussian Splatting

A novel simulation reality closed loop learning framework for autonomous robot skill learning

A Method for Strategic Migration from Simulation to Real Manipulator System.

GRS: Generating Robotic Simulation Tasks from Real-World Images

Sim-and-Real Reinforcement Learning for Manipulation: A Consensus-based Approach

Gen2Sim: Scaling up Robot Learning in Simulation with Generative Models

Sim2real Learning of Obstacle Avoidance for Robotic Manipulators in Uncertain Environments

Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulation

One-shot sim-to-real transfer policy for robotic assembly via reinforcement learning with visual demonstration

Triple Regression for Camera Agnostic Sim2Real Robot Grasping and Manipulation Tasks

Sim-to-real via latent prediction: Transferring visual non-prehensile manipulation policies

Sim-to-Real Transfer with Action Mapping and State Prediction for Robot Motion Control

General-Purpose Sim2Real Protocol for Learning Contact-Rich Manipulation With Marker-Based Visuotactile Sensors