NeuRAD: Neural Rendering for Autonomous Driving

Adam Tonderski,Carl Lindström,Georg Hess,William Ljungbergh,Lennart Svensson,Christoffer Petersson

2024-04-18

Abstract:Neural radiance fields (NeRFs) have gained popularity in the autonomous driving (AD) community. Recent methods show NeRFs' potential for closed-loop simulation, enabling testing of AD systems, and as an advanced training data augmentation technique. However, existing methods often require long training times, dense semantic supervision, or lack generalizability. This, in turn, hinders the application of NeRFs for AD at scale. In this paper, we propose NeuRAD, a robust novel view synthesis method tailored to dynamic AD data. Our method features simple network design, extensive sensor modeling for both camera and lidar -- including rolling shutter, beam divergence and ray dropping -- and is applicable to multiple datasets out of the box. We verify its performance on five popular AD datasets, achieving state-of-the-art performance across the board. To encourage further development, we will openly release the NeuRAD source code. See <a class="link-external link-https" href="https://github.com/georghess/NeuRAD" rel="external noopener nofollow">this https URL</a> .

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The main problems that this paper attempts to solve are the limitations of existing Neural Radiance Fields (NeRFs) methods in the application of the autonomous driving (AD) field. These limitations include:

1. **Long - time training**: Existing NeRFs methods usually require a long time to train, which restricts their application in large - scale closed - loop simulation or data augmentation.
2. **Dense semantic supervision**: Many methods rely on dense semantic labels, which are difficult to obtain in practical applications.
3. **Lack of generalization ability**: Existing methods tend to over - fit specific datasets and cannot generalize well to other datasets.

To solve these problems, the paper proposes NeuRAD (Neural Rendering for Autonomous Driving), which is a new view - synthesis method specifically designed for dynamic autonomous driving scenarios. The main features of NeuRAD include:

- **Simple network design**: NeuRAD adopts a simple but efficient network architecture and can handle large - scale autonomous driving data.
- **Extensive sensor modeling**: NeuRAD not only supports cameras but also LiDAR (Light Detection and Ranging), including features such as rolling shutter, beam divergence and ray loss.
- **Applicable to multiple datasets**: NeuRAD can be directly used on multiple datasets without the need for dataset - specific adjustments.

Through these improvements, NeuRAD can achieve state - of - the - art performance on multiple popular autonomous driving datasets, and has better generalization ability and shorter training time.