Abstract:In the realm of autonomous driving, achieving precise 3D reconstruction of the driving environment is critical for ensuring safety and effective navigation. Neural Radiance Fields (NeRF) have shown promise in creating highly detailed and accurate models of complex environments. However, the application of NeRF in autonomous driving scenarios encounters several challenges, primarily due to the sparsity of viewpoints inherent in camera trajectories and the constraints on data collection in unbounded outdoor scenes, which typically occur along predetermined paths. This limitation not only reduces the available scene information but also poses significant challenges for NeRF training, as the sparse and path-distributed observational data leads to under-representation of the scene's geometry. In this paper, we introduce HarmonicNeRF, a novel approach for outdoor self-supervised monocular scene reconstruction. HarmonicNeRF capitalizes on the strengths of NeRF and enhances surface reconstruction accuracy by augmenting the input space with geometry-informed synthetic views. This is achieved through the application of spherical harmonics to generate novel radiance values, taking into careful consideration the color observations from the limited available real-world views. Additionally, our method incorporates proxy geometry to effectively manage occlusion, generating radiance pseudo-labels that circumvent the limitations of traditional image-warping techniques, which often fail in sparse data conditions typical of autonomous driving environments. Extensive experiments conducted on the KITTI, Argoverse, and NuScenes datasets demonstrate our approach establishes new benchmarks in synthesizing novel depth views and reconstructing scenes, significantly outperforming existing methods. Project page: <a class="link-external link-https" href="https://github.com/Jiawei-Yao0812/HarmonicNeRF" rel="external noopener nofollow">this https URL</a>

DistillNeRF: Perceiving 3D Scenes from Single-Glance Images by Distilling Neural Fields and Foundation Model Features

RD-NERF: Neural Robust Distilled Feature Fields for Sparse-View Scene Segmentation

EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervision

FeatureNeRF: Learning Generalizable NeRFs by Distilling Foundation Models

Drone-NeRF: Efficient NeRF based 3D scene reconstruction for large-scale drone survey

Depth-supervised NeRF: Fewer Views and Faster Training for Free

Large-Scale Neural Scene Disentanglement Approach for Self-Driving Simulation

MonoNeRD: NeRF-like Representations for Monocular 3D Object Detection

HarmonicNeRF: Geometry-Informed Synthetic View Augmentation for 3D Scene Reconstruction in Driving Scenarios

SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fields

GP-NeRF: Generalized Perception NeRF for Context-Aware 3D Scene Understanding

DaRF: Boosting Radiance Fields from Sparse Inputs with Monocular Depth Adaptation

NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fields

PC-NeRF: Parent-Child Neural Radiance Fields under Partial Sensor Data Loss in Autonomous Driving Environments

SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis

NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusion

One-Shot Neural Fields for 3D Object Understanding

Lightning NeRF: Efficient Hybrid Scene Representation for Autonomous Driving

DSSMNeRF: Depth Self-supervised MVS NeRF

OV-NeRF: Open-vocabulary Neural Radiance Fields with Vision and Language Foundation Models for 3D Semantic Understanding

Omni-Recon: Harnessing Image-based Rendering for General-Purpose Neural Radiance Fields