OASim: an Open and Adaptive Simulator based on Neural Rendering for Autonomous Driving

Guohang Yan,Jiahao Pi,Jianfei Guo,Zhaotong Luo,Min Dou,Nianchen Deng,Qiusheng Huang,Daocheng Fu,Licheng Wen,Pinlong Cai,Xing Gao,Xinyu Cai,Bo Zhang,Xuemeng Yang,Yeqi Bai,Hongbin Zhou,Botian Shi

2024-02-06

Abstract:With deep learning and computer vision technology development, autonomous driving provides new solutions to improve traffic safety and efficiency. The importance of building high-quality datasets is self-evident, especially with the rise of end-to-end autonomous driving algorithms in recent years. Data plays a core role in the algorithm closed-loop system. However, collecting real-world data is expensive, time-consuming, and unsafe. With the development of implicit rendering technology and in-depth research on using generative models to produce data at scale, we propose OASim, an open and adaptive simulator and autonomous driving data generator based on implicit neural rendering. It has the following characteristics: (1) High-quality scene reconstruction through neural implicit surface reconstruction technology. (2) Trajectory editing of the ego vehicle and participating vehicles. (3) Rich vehicle model library that can be freely selected and inserted into the scene. (4) Rich sensors model library where you can select specified sensors to generate data. (5) A highly customizable data generation system can generate data according to user needs. We demonstrate the high quality and fidelity of the generated data through perception performance evaluation on the Carla simulator and real-world data acquisition. Code is available at

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper attempts to address the issues of the large data demand and high-quality data generation faced in the development of autonomous driving technology. Specifically, the paper proposes OASim, an open and adaptable autonomous driving simulator and data generator based on neural implicit reconstruction and rendering technology. This simulator aims to solve the following key problems:

1. **Construction of high-quality datasets**: With the development of end-to-end autonomous driving algorithms, data plays a core role in the algorithm's closed-loop system. However, real-world data collection is not only costly and time-consuming but also poses safety risks.

2. **Generation of diverse data and rare scenarios**: Data collected from actual roads is often repetitive and redundant, making it difficult to capture corner cases. These rare but important scenarios are crucial for training complex algorithm models.

3. **Low-cost and efficient testing methods**: Traditional real-world road testing is not only expensive and inefficient but may also face legal restrictions. Simulators provide a low-cost, efficient, and customizable means of testing.

OASim addresses the above issues through the following features:
- **High-fidelity real-time rendering**: Utilizes neural implicit reconstruction technology to achieve high-quality scene reconstruction.
- **Trajectory editing and traffic flow simulation**: Allows editing of the trajectories of the ego vehicle and other vehicles, and simulates interactions within the traffic flow.
- **Rich sensor model library**: Supports various sensor configurations and can generate data as needed.
- **Highly customizable data generation system**: Users can generate specific datasets according to their requirements.

In summary, OASim aims to provide a powerful tool for autonomous driving research through advanced neural implicit reconstruction and rendering technology, thereby accelerating algorithm iteration and improving system accuracy and reliability.