Abstract:In the realm of autonomous driving, achieving precise 3D reconstruction of the driving environment is critical for ensuring safety and effective navigation. Neural Radiance Fields (NeRF) have shown promise in creating highly detailed and accurate models of complex environments. However, the application of NeRF in autonomous driving scenarios encounters several challenges, primarily due to the sparsity of viewpoints inherent in camera trajectories and the constraints on data collection in unbounded outdoor scenes, which typically occur along predetermined paths. This limitation not only reduces the available scene information but also poses significant challenges for NeRF training, as the sparse and path-distributed observational data leads to under-representation of the scene's geometry. In this paper, we introduce HarmonicNeRF, a novel approach for outdoor self-supervised monocular scene reconstruction. HarmonicNeRF capitalizes on the strengths of NeRF and enhances surface reconstruction accuracy by augmenting the input space with geometry-informed synthetic views. This is achieved through the application of spherical harmonics to generate novel radiance values, taking into careful consideration the color observations from the limited available real-world views. Additionally, our method incorporates proxy geometry to effectively manage occlusion, generating radiance pseudo-labels that circumvent the limitations of traditional image-warping techniques, which often fail in sparse data conditions typical of autonomous driving environments. Extensive experiments conducted on the KITTI, Argoverse, and NuScenes datasets demonstrate our approach establishes new benchmarks in synthesizing novel depth views and reconstructing scenes, significantly outperforming existing methods. Project page: <a class="link-external link-https" href="https://github.com/Jiawei-Yao0812/HarmonicNeRF" rel="external noopener nofollow">this https URL</a>

Occlusion-Free Scene Recovery Via Neural Radiance Fields

3D Scene De-occlusion in Neural Radiance Fields: A Framework for Obstacle Removal and Realistic Inpainting

Semantic-aware Occlusion Filtering Neural Radiance Fields in the Wild

An Exploration of Neural Radiance Field Scene Reconstruction: Synthetic, Real-world and Dynamic Scenes

SG-NeRF: Neural Surface Reconstruction with Scene Graph Optimization

SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fields

HarmonicNeRF: Geometry-Informed Synthetic View Augmentation for 3D Scene Reconstruction in Driving Scenarios

Combine Neural Radiance Fields in Different Reference Coordinates

RANRAC: Robust Neural Scene Representations via Random Ray Consensus

Enhance-NeRF: Multiple Performance Evaluation for Neural Radiance Fields

NeRFVS: Neural Radiance Fields for Free View Synthesis Via Geometry Scaffolds

Explicit Composition of Neural Radiance Fields by Learning an Occlusion Field.

Pano-NeRF: Synthesizing High Dynamic Range Novel Views with Geometry from Sparse Low Dynamic Range Panoramic Images

Clean-NeRF: Reformulating NeRF to account for View-Dependent Observations

PC-NeRF: Parent-Child Neural Radiance Fields under Partial Sensor Data Loss in Autonomous Driving Environments

NeRF-HuGS: Improved Neural Radiance Fields in Non-static Scenes Using Heuristics-Guided Segmentation

Hallucinated Neural Radiance Fields in the Wild

Camera and LiDAR Fusion for Urban Scene Reconstruction and Novel View Synthesis via Voxel-Based Neural Radiance Fields

NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild

Enhancing Neural Radiance Fields with Depth and Normal Completion Priors from Sparse Views

Depth-supervised NeRF: Fewer Views and Faster Training for Free