LiDAR-GS:Real-time LiDAR Re-Simulation using Gaussian Splatting

Qifeng Chen,Sheng Yang,Sicong Du,Tao Tang,Peng Chen,Yuchi Huo

2024-10-07

Abstract:LiDAR simulation plays a crucial role in closed-loop simulation for autonomous driving. Although recent advancements, such as the use of reconstructed mesh and Neural Radiance Fields (NeRF), have made progress in simulating the physical properties of LiDAR, these methods have struggled to achieve satisfactory frame rates and rendering quality. To address these limitations, we present LiDAR-GS, the first LiDAR Gaussian Splatting method, for real-time high-fidelity re-simulation of LiDAR sensor scans in public urban road scenes. The vanilla Gaussian Splatting, designed for camera models, cannot be directly applied to LiDAR re-simulation. To bridge the gap between passive camera and active LiDAR, our LiDAR-GS designs a differentiable laser beam splatting, grounded in the LiDAR range view model. This innovation allows for precise surface splatting by projecting lasers onto micro cross-sections, effectively eliminating artifacts associated with local affine approximations. Additionally, LiDAR-GS leverages Neural Gaussian Fields, which further integrate view-dependent clues, to represent key LiDAR properties that are influenced by the incident angle and external factors. Combining these practices with some essential adaptations, e.g., dynamic instances decomposition, our approach succeeds in simultaneously re-simulating depth, intensity, and ray-drop channels, achieving state-of-the-art results in both rendering frame rate and quality on publically available large scene datasets. Our source code will be made publicly available.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to achieve high - fidelity real - time re - simulation of LiDAR sensor frames in an autonomous driving system. Although progress has been made in recent years in using reconstructed meshes and neural radiance fields (NeRF), these methods still face challenges in achieving satisfactory frame rates and rendering quality. To this end, the paper proposes LiDAR - GS, which is the first LiDAR Gaussian point - painting method for real - time high - fidelity re - simulation of LiDAR sensor scans in public urban road scenes.

Specifically, the paper addresses the following key issues:
1. **Differences between passive cameras and active LiDAR**: Traditional Gaussian point - painting methods are designed for camera models and have limitations when directly applied to LiDAR re - simulation. LiDAR - GS achieves accurate surface point - painting by designing a differentiable laser beam point - painting method based on the LiDAR range - view model, effectively eliminating artifacts caused by local affine approximation.
2. **High - fidelity modeling and rendering of LiDAR properties**: LiDAR - GS utilizes neural Gaussian fields to integrate view - dependent cues to represent key LiDAR properties affected by the incident angle and other external factors.
3. **Simulation of laser beam loss characteristics**: To simulate laser beam loss characteristics, LiDAR - GS assigns a learnable laser beam loss probability to each Gaussian primitive, enabling forward rendering to determine the effectiveness of the corresponding emitted laser beam for each range - view image pixel.
4. **Significant differences in vertical and horizontal angular resolutions**: LiDAR - GS proposes a regularization term and a compact axis - aligned bounding box (AABB) strategy to constrain the scale, pixel coverage, and ray scattering angle of Gaussian primitives, ensuring accurate gradient propagation.

By combining the above methods and strategies, LiDAR - GS has achieved the state - of - the - art in both rendering quality and efficiency on public datasets. In particular, in terms of dynamic instance handling, by decomposing the reconstruction and re - placing dynamic instances into the static background, the ability of scene editing is further enhanced.