Abstract:Active single-photon light detection and ranging (LiDAR) imaging technology adopting emerging single-photon avalanche diode (SPAD) has been used extensively for 3D imaging of complex scenes. However, the different signal-to-noise data acquired by SPAD under complex scenes and the larger pixel size of the SPAD pose a great challenge for robustly estimating the scene depth with rich detailed information. In this paper, we have designed a two-stage network including a multi-scale encoder-decoder sub-network and an intensity-guided edge refinement sub-network using a sensor fusion strategy to improve the robustness of the depth estimation network. In the first stage of the sub-network, we use the UNet3+ network to integrate fine-grained detail information and coarse-grained semantic information from full-scale through skip connections across scales on the decoder. In the meantime, the deep boosting network is added to the encoder to extract rich contextual information from the spatial-temporal domain to alleviate semantic gaps between the encoder and decoder. Moreover, in the second stage of the sub-network, we adopt an inverted y-shaped backbone network with the same structure between the encoder and decoder coupled with the same weight at both encoders. The purpose of this design is to extract structural similarity features with spatial consistency from intensity and depth to compensate for the missing detail of reconstructed depth by the first stage sub-network. The experimental results conducted on both simulated and real-world datasets demonstrate that our network outperforms previous networks. In addition, networks are trained on single or multiple signal-to-background ratio datasets to estimate the real-world data respectively, and the results show that our proposed network is more robust.

Efficient Active Depth Sensing by Laser Speckle Projection System

Guided depth enhancement via a fast marching method

A New Approach of Point Cloud Processing and Scene Segmentation for Guiding the Visually Impaired

Depth Estimation for Speckle Projection System Using Progressive Reliable Points Growing Matching.

An Efficient High-Throughput Structured-Light Depth Engine

Expanding Sparse LiDAR Depth and Guiding Stereo Matching for Robust Dense Depth Estimation

Depth map enhancement based on color and depth consistency

Multi-channel Residual Network Model for Accurate Estimation of Spatially-Varying and Depth-Dependent Defocus Kernels.

Accurate 3D reconstruction of single-frame speckle-encoded textureless surfaces based on densely connected stereo matching network

High-Quality Depth Recovery Via Interactive Multi-view Stereo

Sparse LIDAR Measurement Fusion with Joint Updating Cost for Fast Stereo Matching

Accurate Depth Sensing by Filtering IR Cost on the Guide of RGB Information

A Light Field Depth Estimation Algorithm by Edge Disparity Extraction

High-Dynamic-Range 3-D Shape Measurement With Adaptive Speckle Projection Through Segmentation-Based Mapping

Robust single-photon 3D imaging based on full-scale feature integration and intensity edge guidance

Single-photon lidar depth imaging based on sparsity adaptive matching pursuit method

Single-shot 3D Shape Measurement Using an End-to-end Stereo Matching Network for Speckle Projection Profilometry.

A Self-Calibrated Photo-Geometric Depth Camera

Three 1-bit Speckle-embedded pulse-width modulation patterns for robust absolute 3D measurement

Intensity-guided Depth Image Estimation in Long-Range Lidar

Dense depth acquisition via one-shot stripe structured light