Abstract:We consider the problem of few-viewpoint 3D surface reconstruction using raw measurements from a lidar system. Lidar captures 3D scene geometry by emitting pulses of light to a target and recording the speed-of-light time delay of the reflected light. However, conventional lidar systems do not output the raw, captured waveforms of backscattered light; instead, they pre-process these data into a 3D point cloud. Since this procedure typically does not accurately model the noise statistics of the system, exploit spatial priors, or incorporate information about downstream tasks, it ultimately discards useful information that is encoded in raw measurements of backscattered light. Here, we propose to leverage raw measurements captured with a single-photon lidar system from multiple viewpoints to optimize a neural surface representation of a scene. The measurements consist of time-resolved photon count histograms, or transients, which capture information about backscattered light at picosecond time scales. Additionally, we develop new regularization strategies that improve robustness to photon noise, enabling accurate surface reconstruction with as few as 10 photons per pixel. Our method outperforms other techniques for few-viewpoint 3D reconstruction based on depth maps, point clouds, or conventional lidar as demonstrated in simulation and with captured data.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to perform high - quality 3D surface reconstruction using the raw measurement data of single - photon Lidar (Light Detection and Ranging) with only a few viewpoints (for example, 2 to 5 viewpoints). Specifically, traditional Lidar systems usually pre - process the captured raw waveform data into 3D point clouds, which will lead to the loss of useful information. Therefore, this paper proposes a new method - **Transientangelo**, aiming to directly use the single - photon Lidar raw measurement data (i.e., time - resolved photon - count histograms or transients) captured from multiple viewpoints, and recover a higher - quality surface by optimizing the scene representation based on the signed distance function (SDF). ### Main problems 1. **Challenges of 3D surface reconstruction in the case of sparse multi - viewpoints**: When the number of input viewpoints is very limited, traditional image - based or multi - viewpoint - based methods are difficult to provide accurate 3D reconstruction results. 2. **Noise robustness in the case of low photon counts**: In the case of low photon counts (10 - 300 photons per pixel), how to ensure the accuracy of the reconstruction results is an important challenge. 3. **Making full use of the original Lidar data**: Traditional Lidar systems usually discard useful original measurement information. How to effectively use this information to improve the reconstruction quality is another key issue. ### Solutions To address the above challenges, the paper proposes the following innovations: 1. **Neural surface representation based on transient supervision**: Use transient data (time - resolved photon - count histograms) as a supervision signal, combined with a neural network to optimize the SDF representation of the scene. 2. **Regularization techniques**: Introduce new regularization strategies, such as weight variance regularization and reflectivity loss, to improve the robustness to low - photon - count data and generate smoother and more accurate surfaces. 3. **Multi - viewpoint simulation and real - data sets**: Construct a multi - viewpoint transient data set with different photon - count levels to verify the effectiveness of the method. ### Method overview - **Measurement model**: Describe the measurement process of single - photon Lidar, including the modeling of photon - count histograms. - **Surface - based transient rendering**: Combine the neural network representation and the time - resolved volume - rendering equation to render the transient. - **Optimization objective**: Define the training loss function, including transient loss, reflectivity loss, weight variance regularization, etc. - **Implementation details**: Explain in detail the implementation and training process of the model, including the optimizer used and the learning - rate scheduling strategy. Through these methods, Transientangelo can recover a high - fidelity 3D surface from the original Lidar data with only a few viewpoints, and it performs especially well under low - photon - count conditions.

Transientangelo: Few-Viewpoint Surface Reconstruction Using Single-Photon Lidar

Three Dimensional Surface Reconstruction Based on Light Field Imaging

Transient Neural Radiance Fields for Lidar View Synthesis and 3D Reconstruction

Improved Fast 3D Reconstruction Algorithm Based on Laser Scanning Point Cloud

Real-time 3D reconstruction from single-photon lidar data using plug-and-play point cloud denoisers

Sketched RT3D: How to reconstruct billions of photons per second

Bayesian 3D Reconstruction of Subsampled Multispectral Single-photon Lidar Signals

EM-based approach to 3D reconstruction from single-waveform multispectral Lidar data

A robust real-time surface reconstruction method on point clouds captured from a 3D surface photogrammetry system

PlatoNeRF: 3D Reconstruction in Plato's Cave via Single-View Two-Bounce Lidar

LiDAR4D: Dynamic Neural Fields for Novel Space-time View LiDAR Synthesis

Non-line-of-sight snapshots and background mapping with an active corner camera

A novel rapid point-cloud surface reconstruction algorithm for laser imaging radar

A photon-efficient method based on curve fitting for photon counting 3D imaging lidar

Approach for LIDAR signals with multiple returns

Few-shot Non-line-of-sight Imaging with Signal-surface Collaborative Regularization

Non-line-of-Sight Imaging via Neural Transient Fields

Robust and Guided Bayesian Reconstruction of Single-Photon 3D Lidar Data: Application to Multispectral and Underwater Imaging

Airborne Single-Photon LiDAR Towards a Small-Sized and Low-Power Payload

3D Reconstruction & Assessment Framework based on affordable 2D Lidar

Efficient Implicit Neural Reconstruction Using LiDAR