Abstract:Vision-based object detectors are a crucial basis for robotics applications as they provide valuable information about object localisation in the environment. These need to ensure high reliability in different lighting conditions, occlusions, and visual artifacts, all while running in real-time. Collecting and annotating real-world data for these networks is prohibitively time consuming and costly, especially for custom assets, such as industrial objects, making it untenable for generalization to in-the-wild scenarios. To this end, we present Synthetica, a method for large-scale synthetic data generation for training robust state estimators. This paper focuses on the task of object detection, an important problem which can serve as the front-end for most state estimation problems, such as pose estimation. Leveraging data from a photorealistic ray-tracing renderer, we scale up data generation, generating 2.7 million images, to train highly accurate real-time detection transformers. We present a collection of rendering randomization and training-time data augmentation techniques conducive to robust sim-to-real performance for vision tasks. We demonstrate state-of-the-art performance on the task of object detection while having detectors that run at 50-100Hz which is 9 times faster than the prior SOTA. We further demonstrate the usefulness of our training methodology for robotics applications by showcasing a pipeline for use in the real world with custom objects for which there do not exist prior datasets. Our work highlights the importance of scaling synthetic data generation for robust sim-to-real transfer while achieving the fastest real-time inference speeds. Videos and supplementary information can be found at this URL: <a class="link-external link-https" href="https://sites.google.com/view/synthetica-vision" rel="external noopener nofollow">this https URL</a>.

SynthDa: Exploiting Existing Real-World Data for Usable and Accessible Synthetic Data Generation

Synthetic Data for Deep Learning

Artificial Data, Real Insights: Evaluating Opportunities and Risks of Expanding the Data Ecosystem with Synthetic Data

Enabling Synthetic Data adoption in regulated domains

Comprehensive Exploration of Synthetic Data Generation: A Survey

Synthetic Data in Human Analysis: A Survey

Synthetica: Large Scale Synthetic Data for Robot Perception

Indoor synthetic data generation: A systematic review

Boosting Data Analytics With Synthetic Volume Expansion

On the Equivalency, Substitutability, and Flexibility of Synthetic Data

Exploring the Impact of Synthetic Data for Aerial-view Human Detection

Beyond Privacy: Navigating the Opportunities and Challenges of Synthetic Data

AnySynth: Harnessing the Power of Image Synthetic Data Generation for Generalized Vision-Language Tasks

SynSys: A Synthetic Data Generation System for Healthcare Applications

SynFace: Face Recognition with Synthetic Data

Training Deep Face Recognition Systems with Synthetic Data

Efficient Realistic Data Generation Framework leveraging Deep Learning-based Human Digitization

Scalable Modular Synthetic Data Generation for Advancing Aerial Autonomy

Best Practices and Lessons Learned on Synthetic Data

The Use of Synthetic Data to Train AI Models: Opportunities and Risks for Sustainable Development

Learning to Generate Synthetic Data via Compositing