Abstract:In the realm of autonomous driving, achieving precise 3D reconstruction of the driving environment is critical for ensuring safety and effective navigation. Neural Radiance Fields (NeRF) have shown promise in creating highly detailed and accurate models of complex environments. However, the application of NeRF in autonomous driving scenarios encounters several challenges, primarily due to the sparsity of viewpoints inherent in camera trajectories and the constraints on data collection in unbounded outdoor scenes, which typically occur along predetermined paths. This limitation not only reduces the available scene information but also poses significant challenges for NeRF training, as the sparse and path-distributed observational data leads to under-representation of the scene's geometry. In this paper, we introduce HarmonicNeRF, a novel approach for outdoor self-supervised monocular scene reconstruction. HarmonicNeRF capitalizes on the strengths of NeRF and enhances surface reconstruction accuracy by augmenting the input space with geometry-informed synthetic views. This is achieved through the application of spherical harmonics to generate novel radiance values, taking into careful consideration the color observations from the limited available real-world views. Additionally, our method incorporates proxy geometry to effectively manage occlusion, generating radiance pseudo-labels that circumvent the limitations of traditional image-warping techniques, which often fail in sparse data conditions typical of autonomous driving environments. Extensive experiments conducted on the KITTI, Argoverse, and NuScenes datasets demonstrate our approach establishes new benchmarks in synthesizing novel depth views and reconstructing scenes, significantly outperforming existing methods. Project page: <a class="link-external link-https" href="https://github.com/Jiawei-Yao0812/HarmonicNeRF" rel="external noopener nofollow">this https URL</a>

NeO 360: Neural Fields for Sparse View Synthesis of Outdoor Scenes

Learning Spherical Radiance Field for Efficient 360° Unbounded Novel View Synthesis

Neural Rendering in a Room: Amodal 3D Understanding and Free-Viewpoint Rendering for the Closed Scene Composed of Pre-Captured Objects

Neural Radiance Fields From Sparse RGB-D Images for High-Quality View Synthesis

Non-uniform Sampling Strategies for NeRF on 360{\textdegree} images

Neural 3D Video Synthesis from Multi-view Video

PERF: Panoramic Neural Radiance Field From a Single Panorama

A Large-Scale Outdoor Multi-modal Dataset and Benchmark for Novel View Synthesis and Implicit Scene Reconstruction

StructNeRF: Neural Radiance Fields for Indoor Scenes With Structural Hints

SparseGS: Real-Time 360° Sparse View Synthesis using Gaussian Splatting

NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360° Views

NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild

Multi-view stereo-regulated NeRF for urban scene novel view synthesis

wildNeRF: Complete view synthesis of in-the-wild dynamic scenes captured using sparse monocular data

Collaborative neural radiance fields for novel view synthesis

D-NPC: Dynamic Neural Point Clouds for Non-Rigid View Synthesis from Monocular Video

ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Image

SNeRF: Stylized Neural Implicit Representations for 3D Scenes

Multi-View Stereo and Depth Priors Guided NeRF for View Synthesis.

HarmonicNeRF: Geometry-Informed Synthetic View Augmentation for 3D Scene Reconstruction in Driving Scenarios

ViewNeRF: Unsupervised Viewpoint Estimation Using Category-Level Neural Radiance Fields