Abstract:Depth completion aims to recover dense depth maps from sparse depth maps. Recent approaches have used additional modalities as guidance to improve depth completion performance. Image-guided depth completion uses scene information from color images, but it still produces inaccurate object boundaries. In this paper, we propose deep sparse depth completion using multi-scale residuals and channel shuffle, named ReCSNet. ReCSNet is a dual-branch network based on a U-shaped architecture. ReCSNet consists of one VIS-Semantic-Guided Branch (VSGB) and one Sparse Depth Guided Branch (SDGB) to get global color, edge information, and local accurate depth information. VSGB utilizes two encoders to extract features from the VIS-Semantic image pairs and the sparse depth maps, and employs a feature channel shuffle mechanism to blend the two sets of encoded features. The semi-dense depth map generated by VSGB is concatenated with the original sparse depth map and input into SDGB to predict the second semi-dense depth map. The confidence maps generated by the two branches are adaptively fused to generate the final depth map. Moreover, we incorporate multi-scale residuals obtained from the VIS image and concatenate them with the decoded features to further enhance the constraint on object boundaries. At the rear of the dual-branch network, we add a Repetitive Deformable Convolution Module (RDCM) to further refine the depth values in object edges. Experimental results show that ReCSNet achieves outstanding performance on the KITTI depth completion validation dataset with an improvement of 16mm in the root mean square error (RMSE) metric.

Deep Sparse Depth Completion Using Joint Depth and Normal Estimation.

Least Square Estimation Network for Depth Completion

Depth Completion from Sparse LiDAR Data with Depth-Normal Constraints

MFF-Net: Towards Efficient Monocular Depth Completion With Multi-Modal Feature Fusion

DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image

Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera

DenseLiDAR: A Real-Time Pseudo Dense Depth Guided Depth Completion Network

NDDepth: Normal-Distance Assisted Monocular Depth Estimation and Completion

Deep Depth Completion from Extremely Sparse Data: A Survey

Deep Sparse Depth Completion Using Multi-Scale Residuals and Channel Shuffle

Deep Sparse Depth Completion Using Multi-Affinity Matrix

Learning Guided Convolutional Network for Depth Completion

Depth Map Completion by Jointly Exploiting Blurry Color Images and Sparse Depth Maps

Depth Completion Towards Different Sensor Configurations Via Relative Depth Map Estimation and Scale Recovery

Self‐supervised Depth Completion with Multi‐view Geometric Constraints

SGTBN: Generating Dense Depth Maps from Single-Line LiDAR

Depth-Independent Depth Completion via Least Square Estimation

PENet: Towards Precise and Efficient Image Guided Depth Completion

Deterministic Guided LiDAR Depth Map Completion

Deep Depth Estimation from Visual-Inertial SLAM