Abstract:In the realm of autonomous driving, achieving precise 3D reconstruction of the driving environment is critical for ensuring safety and effective navigation. Neural Radiance Fields (NeRF) have shown promise in creating highly detailed and accurate models of complex environments. However, the application of NeRF in autonomous driving scenarios encounters several challenges, primarily due to the sparsity of viewpoints inherent in camera trajectories and the constraints on data collection in unbounded outdoor scenes, which typically occur along predetermined paths. This limitation not only reduces the available scene information but also poses significant challenges for NeRF training, as the sparse and path-distributed observational data leads to under-representation of the scene's geometry. In this paper, we introduce HarmonicNeRF, a novel approach for outdoor self-supervised monocular scene reconstruction. HarmonicNeRF capitalizes on the strengths of NeRF and enhances surface reconstruction accuracy by augmenting the input space with geometry-informed synthetic views. This is achieved through the application of spherical harmonics to generate novel radiance values, taking into careful consideration the color observations from the limited available real-world views. Additionally, our method incorporates proxy geometry to effectively manage occlusion, generating radiance pseudo-labels that circumvent the limitations of traditional image-warping techniques, which often fail in sparse data conditions typical of autonomous driving environments. Extensive experiments conducted on the KITTI, Argoverse, and NuScenes datasets demonstrate our approach establishes new benchmarks in synthesizing novel depth views and reconstructing scenes, significantly outperforming existing methods. Project page: <a class="link-external link-https" href="https://github.com/Jiawei-Yao0812/HarmonicNeRF" rel="external noopener nofollow">this https URL</a>

DrivingRecon: Large 4D Gaussian Reconstruction Model For Autonomous Driving

DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes

Driv3R: Learning Dense 4D Reconstruction for Autonomous Driving

ReconDreamer: Crafting World Models for Driving Scene Reconstruction via Online Restoration

DrivingForward: Feed-forward 3D Gaussian Splatting for Driving Scene Reconstruction from Flexible Surround-view Input

UniGaussian: Driving Scene Reconstruction from Multiple Camera Models via Unified Gaussian Representations

DreamDrive: Generative 4D Scene Modeling from Street View Images

S^3Gaussian: Self-Supervised Street Gaussians for Autonomous Driving

$\textit{S}^3$Gaussian: Self-Supervised Street Gaussians for Autonomous Driving

Self-driving Simulation Scene Reconstruction Using Self-Supervised Depth Completion.

DeSiRe-GS: 4D Street Gaussians for Static-Dynamic Decomposition and Surface Reconstruction for Urban Driving Scenes

OmniRe: Omni Urban Scene Reconstruction

Large-Scale Neural Scene Disentanglement Approach for Self-Driving Simulation

Visualization pipeline of autonomous driving scenes based on FCCR-3D reconstruction

DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representation

DGTR: Distributed Gaussian Turbo-Reconstruction for Sparse-View Vast Scenes

LiHi-GS: LiDAR-Supervised Gaussian Splatting for Highway Driving Scene Reconstruction

Projecting Your View Attentively: Monocular Road Scene Layout Estimation Via Cross-view Transformation

Omni-Scene: Omni-Gaussian Representation for Ego-Centric Sparse-View Scene Reconstruction

AutoSplat: Constrained Gaussian Splatting for Autonomous Driving Scene Reconstruction

HarmonicNeRF: Geometry-Informed Synthetic View Augmentation for 3D Scene Reconstruction in Driving Scenarios