Abstract:Sim2real, that is, the transfer of learned control policies from simulation to real world, is an area of growing interest in robotics due to its potential to efficiently handle complex tasks. The sim2real approach faces challenges due to mismatches between simulation and reality. These discrepancies arise from inaccuracies in modeling physical phenomena and asynchronous control, among other factors. To this end, we introduce EAGERx, a framework with a unified software pipeline for both real and simulated robot learning. It can support various simulators and aids in integrating state, action and time-scale abstractions to facilitate learning. EAGERx's integrated delay simulation, domain randomization features, and proposed synchronization algorithm contribute to narrowing the sim2real gap. We demonstrate (in the context of robot learning and beyond) the efficacy of EAGERx in accommodating diverse robotic systems and maintaining consistent simulation behavior. EAGERx is open source and its code is available at <a class="link-external link-https" href="https://eagerx.readthedocs.io" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve This paper aims to address the **Sim2real transfer problem** in the field of robotics. Specifically, the research team has developed a new framework called EAGERx to tackle the challenges faced when transferring control strategies from simulation environments to the real world. #### Main Problems and Challenges 1. **Gap between simulation and reality**: Significant differences exist between simulation environments and the real world due to inaccurate physical modeling, asynchronous control, and other issues. 2. **Compatibility with different simulators**: Existing Sim2real frameworks typically support only specific simulators (e.g., Gazebo), limiting flexibility. 3. **Synchronization issues between components**: In simulations faster than real-time, existing frameworks struggle to maintain synchronization between components, leading to system instability. #### Main Contributions of EAGERx 1. **Synchronization protocol**: Ensures consistent simulation behavior even at speeds faster than real-time (C1). 2. **Modular design**: Supports various robotic systems and their state, action, and time scale abstractions (C2). 3. **Multi-engine compatibility**: Capable of being compatible with multiple simulation engines (C3). 4. **Latency simulation and domain randomization**: Integrates latency simulation and domain randomization to bridge the gap between simulation and reality (C4). ### Experimental Validation To validate the aforementioned contributions, the research team conducted several experiments: - **Swing-up task**: Demonstrates the importance of latency compensation in reinforcement learning and performs zero-shot evaluation. - **Box-pushing task**: Uses the Viper 300x robotic arm to emphasize the importance of domain randomization. - **Tilt-landing task**: Showcases the integration capability of multiple mobile robots in dynamic tasks. Through these experiments, the paper demonstrates the flexibility and effectiveness of the EAGERx framework in various scenarios.

EAGERx: Graph-Based Framework for Sim2real Robot Learning

Robot Simulation and Reinforcement Learning Training Platform Based on Distributed Architecture.

A novel simulation reality closed loop learning framework for autonomous robot skill learning

RoboGSim: A Real2Sim2Real Robotic Gaussian Splatting Simulator

i-Sim2Real: Reinforcement Learning of Robotic Policies in Tight Human-Robot Interaction Loops

Sim-to-Real Transfer Learning using Robustified Controllers in Robotic Tasks involving Complex Dynamics

Sim-to-Real Transfer with Neural-Augmented Robot Simulation

NeuronsGym: A Hybrid Framework and Benchmark for Robot Tasks with Sim2Real Policy Learning

IRASim: Learning Interactive Real-Robot Action Simulators

Dynamics as Prompts: In-Context Learning for Sim-to-Real System Identifications

Gen2Sim: Scaling up Robot Learning in Simulation with Generative Models

RL-GSBridge: 3D Gaussian Splatting Based Real2Sim2Real Method for Robotic Manipulation Learning

Sim2real Learning of Obstacle Avoidance for Robotic Manipulators in Uncertain Environments

REAL-X -- Robot open-Ended Autonomous Learning Architectures: Achieving Truly End-to-End Sensorimotor Autonomous Learning Systems

Real–Sim–Real Transfer for Real-World Robot Control Policy Learning with Deep Reinforcement Learning

An Open-source Sim2Real Approach for Sensor-independent Robot Navigation in a Grid

A Mobile Robot Experiment System with Lightweight Simulator Generator for Deep Reinforcement Learning Algorithm.

Generalized Robot Learning Framework

Real2Sim or Sim2Real: Robotics Visual Insertion using Deep Reinforcement Learning and Real2Sim Policy Adaptation

DrEureka: Language Model Guided Sim-To-Real Transfer

Close the Sim2real Gap via Physically-based Structured Light Synthetic Data Simulation