Abstract:We present a novel multi-view implicit surface reconstruction technique, termed StreetSurf, that is readily applicable to street view images in widely-used autonomous driving datasets, such as Waymo-perception sequences, without necessarily requiring LiDAR data. As neural rendering research expands rapidly, its integration into street views has started to draw interests. Existing approaches on street views either mainly focus on novel view synthesis with little exploration of the scene geometry, or rely heavily on dense LiDAR data when investigating reconstruction. Neither of them investigates multi-view implicit surface reconstruction, especially under settings without LiDAR data. Our method extends prior object-centric neural surface reconstruction techniques to address the unique challenges posed by the unbounded street views that are captured with non-object-centric, long and narrow camera trajectories. We delimit the unbounded space into three parts, close-range, distant-view and sky, with aligned cuboid boundaries, and adapt cuboid/hyper-cuboid hash-grids along with road-surface initialization scheme for finer and disentangled representation. To further address the geometric errors arising from textureless regions and insufficient viewing angles, we adopt geometric priors that are estimated using general purpose monocular models. Coupled with our implementation of efficient and fine-grained multi-stage ray marching strategy, we achieve state of the art reconstruction quality in both geometry and appearance within only one to two hours of training time with a single RTX3090 GPU for each street view sequence. Furthermore, we demonstrate that the reconstructed implicit surfaces have rich potential for various downstream tasks, including ray tracing and LiDAR simulation.

Drivable Road Reconstruction for Intelligent Vehicles Based on Two-View Geometry

Integrating Visual and Range Data for Road Detection.

Recursive Drivable Road Detection with Shadows Based on Two-Camera Systems

3-D Surround View for Advanced Driver Assistance Systems.

A Reinforced Road Detection Method in Complicated Environment

Improved Mean Shift Algorithm and Its Application

RoadBEV: Road Surface Reconstruction in Bird's Eye View

Road Scene Simulation Based on Vehicle Sensors: an Intelligent Framework Using Random Walk Detection and Scene Stage Reconstruction.

Roadside HD Map Object Reconstruction Using Monocular Camera

DrivingRecon: Large 4D Gaussian Reconstruction Model For Autonomous Driving

Monocular BEV Perception of Road Scenes Via Front-to-Top View Projection

3D RECONSTRUCTION BASED ON 2D VIEW FEATURE

StreetSurf: Extending Multi-view Implicit Surface Reconstruction to Street Views

Automatic roadway geometry measurement algorithm using video images

Spatiotemporal Road Scene Reconstruction Using Superpixel-Based Markov Random Field.

Sequential Monocular Road Detection by Fusing Appearance and Geometric Information

Probabilistic Inference for Occluded and Multiview On-road Vehicle Detection

Structure-aware Roadview Synthesis for Testing Autonomous Perception

MagicDrive: Street View Generation with Diverse 3D Geometry Control

Application of Image Processing and Three-Dimensional Data Reconstruction Algorithm Based on Traffic Video in Vehicle Component Detection

A Joint 3D-2D based Method for Free Space Detection on Roads