Benchmarking the Sim-to-Real Gap in Cloth Manipulation

David Blanco-Mulero,Oriol Barbany,Gokhan Alcan,Adrià Colomé,Carme Torras,Ville Kyrki

2024-01-25

Abstract:Realistic physics engines play a crucial role for learning to manipulate deformable objects such as garments in simulation. By doing so, researchers can circumvent challenges such as sensing the deformation of the object in the realworld. In spite of the extensive use of simulations for this task, few works have evaluated the reality gap between deformable object simulators and real-world data. We present a benchmark dataset to evaluate the sim-to-real gap in cloth manipulation. The dataset is collected by performing a dynamic as well as a quasi-static cloth manipulation task involving contact with a rigid table. We use the dataset to evaluate the reality gap, computational time, and simulation stability of four popular deformable object simulators: MuJoCo, Bullet, Flex, and SOFA. Additionally, we discuss the benefits and drawbacks of each simulator. The benchmark dataset is open-source. Supplementary material, videos, and code, can be found at <a class="link-external link-https" href="https://sites.google.com/view/cloth-sim2real-benchmark" rel="external noopener nofollow">this https URL</a>.

Robotics,Computer Vision and Pattern Recognition,Machine Learning

What problem does this paper attempt to address?

The paper primarily aims to address the sim-to-real gap in cloth manipulation tasks. Specifically:

1. **Research Background**: Cloth manipulation has significant applications in various fields such as caregiving, household chores, and the textile industry. However, due to the infinite degrees of freedom (DoFs) and complex dynamics of deformable objects like cloth, it becomes extremely challenging for robots to handle them.

2. **Role of Simulation Engines**: Current research often relies on physical simulation engines to simulate the cloth manipulation process. This approach can avoid potential safety issues in real-world operations and allows for extensive experimentation in a simulated environment to optimize control strategies. However, these simulation engines have certain inaccuracies in modeling real-world dynamics, known as the "sim-to-real gap."

3. **Contributions of the Paper**:
- Proposed a benchmark dataset to evaluate the performance of different simulation engines in cloth manipulation tasks, particularly for dynamic cloth manipulation.
- Conducted a detailed comparative analysis of four mainstream physical simulation engines (MuJoCo, Bullet, Flex, and SOFA), including their advantages and disadvantages.
- Designed dynamic and quasi-static cloth manipulation tasks, using point cloud data and RGB-D images to assess the gap between the simulation environment and the real world.
- Investigated the relationship between stability, computation time, and the sim-to-real gap at different simulation frequencies.

In summary, this paper aims to establish a comprehensive benchmarking framework to quantify the performance of different simulation engines in cloth manipulation tasks and provides a reference for further improvement of these simulation tools.