Abstract:Raw depth maps captured by depth sensors generally contain missing contents due to glossy, transparent, and sparsity problems. Recent methods well completed flat regions of raw depth maps; however, ignored the accuracy of depth structures. In this paper, an effective depth structure completion method is de-veloped to infer missing depth structures. First, a raw depth map is divided into flat regions and depth structures based on a structure prediction network. Second, two local features including surface normals and Gaussian weights are extracted from a reference RGB image to impose constraints on flat regions and depth structures, separately. Third, a kernel least-square module is adopted to handle the texture-copy ar-tifacts problem. Finally, an iterative optimization model is developed by embedding the two constraints into a Markov random field. The cost function of the model comprises three terms, which limit data fi-delity between completed depth map and raw depth map, smoothness of flat regions, and accuracy of depth structures, respectively. The proposed method is evaluated on four indoor datasets including Mat-terport3D, RealSense, ScanNet, and NYUv2, and compared with eight recent baselines. Quantitative results demonstrate that RMSE and MAE of completed depth maps are considerably reduced by 22.0% and 45.3%, respectively. Visual results show the superiority in completing depth structures and suppressing texture -copy artifacts. Generalization test verify the effectiveness on unseen datasets. (c) 2023 Elsevier Ltd. All rights reserved.

Depth Completion using Piecewise Planar Model

Aggregating Feature Point Cloud for Depth Completion

Least Square Estimation Network for Depth Completion

Depth Map Inpainting Using a Fully Convolutional Network

Efficient Depth Completion Using Learned Bases

Depth map enhancement based on color and depth consistency

Depth Map Completion by Jointly Exploiting Blurry Color Images and Sparse Depth Maps

P3Depth: Monocular Depth Estimation with a Piecewise Planarity Prior

Multi-View Depth Map Sampling for 3D Reconstruction of Natural Scene

Depth Completion with Edge Optimization for Depth Maps from Lidar

Reconstructing Sparsely Represented High-resolution Depth Map from Corrupted Samples

CRF-Based Reconstruction from Narrow-Baseline Image Sequences.

Depth Completion from Sparse LiDAR Data with Depth-Normal Constraints

Depth Completion Towards Different Sensor Configurations Via Relative Depth Map Estimation and Scale Recovery

Depth Upsampling Method Via Markov Random Fields Without Edge-Misaligned Artifacts

NDDepth: Normal-Distance Assisted Monocular Depth Estimation and Completion

Depth Completion Using a View-constrained Deep Prior

Occlusion-Aware Depth Estimation with Adaptive Normal Constraints

Surface Normal and Gaussian Weight Constraints for Indoor Depth Structure Completion.

Monocular Piecewise Depth Estimation in Dynamic Scenes by Exploiting Superpixel Relations