Abstract:We present a novel multi-view implicit surface reconstruction technique, termed StreetSurf, that is readily applicable to street view images in widely-used autonomous driving datasets, such as Waymo-perception sequences, without necessarily requiring LiDAR data. As neural rendering research expands rapidly, its integration into street views has started to draw interests. Existing approaches on street views either mainly focus on novel view synthesis with little exploration of the scene geometry, or rely heavily on dense LiDAR data when investigating reconstruction. Neither of them investigates multi-view implicit surface reconstruction, especially under settings without LiDAR data. Our method extends prior object-centric neural surface reconstruction techniques to address the unique challenges posed by the unbounded street views that are captured with non-object-centric, long and narrow camera trajectories. We delimit the unbounded space into three parts, close-range, distant-view and sky, with aligned cuboid boundaries, and adapt cuboid/hyper-cuboid hash-grids along with road-surface initialization scheme for finer and disentangled representation. To further address the geometric errors arising from textureless regions and insufficient viewing angles, we adopt geometric priors that are estimated using general purpose monocular models. Coupled with our implementation of efficient and fine-grained multi-stage ray marching strategy, we achieve state of the art reconstruction quality in both geometry and appearance within only one to two hours of training time with a single RTX3090 GPU for each street view sequence. Furthermore, we demonstrate that the reconstructed implicit surfaces have rich potential for various downstream tasks, including ray tracing and LiDAR simulation.

MD-Surf: Multimodal Neural Surface Reconstruction from Driving Views

StreetSurf: Extending Multi-view Implicit Surface Reconstruction to Street Views

NeuSurf: On-Surface Priors for Neural Surface Reconstruction from Sparse Input Views

UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction

Sur2f: A Hybrid Representation for High-Quality and Efficient Surface Reconstruction from Multi-view Images

GSurf: 3D Reconstruction via Signed Distance Fields with Direct Gaussian Supervision

TriSpaSurf: A Triple-View Outline Guided 3D Surface Reconstruction of Vehicles from Sparse Point Cloud

DP-MVS: Detail Preserving Multi-View Surface Reconstruction of Large-Scale Scenes

StreetSurfGS: Scalable Urban Street Surface Reconstruction with Planar-based Gaussian Splatting

Surface-Centric Modeling for High-Fidelity Generalizable Neural Surface Reconstruction

MegaSurf: Scalable Large Scene Neural Surface Reconstruction

HelixSurf: A Robust and Efficient Neural Implicit Surface Learning of Indoor Scenes with Iterative Intertwined Regularization

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

MV-DeepSDF: Implicit Modeling with Multi-Sweep Point Clouds for 3D Vehicle Reconstruction in Autonomous Driving

NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction

Dense 3D Surface Reconstruction of Large-Scale Streetscape from Vehicle-Borne Imagery and LiDAR

PR-NeuS: A Prior-based Residual Learning Paradigm for Fast Multi-view Neural Surface Reconstruction

GO-Surf: Neural Feature Grid Optimization for Fast, High-Fidelity RGB-D Surface Reconstruction

Geometry-Guided Neural Implicit Surface Reconstruction

Geo-Neus: Geometry-Consistent Neural Implicit Surfaces Learning for Multi-view Reconstruction

GenS: Generalizable Neural Surface Reconstruction from Multi-View Images