Abstract:Many computer vision tasks, such as monocular depth estimation and height estimation from a satellite orthophoto, have a common underlying goal, which is regression of dense continuous values for the pixels given a single image. We define them as dense continuous-value regression (DCR) tasks. Recent approaches based on deep convolutional neural networks significantly improve the performance of DCR tasks, particularly on pixelwise regression accuracy. However, it still remains challenging to simultaneously preserve the global structure and fine object details in complex scenes. In this article, we take advantage of the efficiency of Laplacian pyramid on representing multiscale contents to reconstruct high-quality signals for complex scenes. We design a Laplacian pyramid neural network (LAPNet), which consists of a Laplacian pyramid decoder (LPD) for signal reconstruction and an adaptive dense feature fusion (ADFF) module to fuse features from the input image. More specifically, we build an LPD to effectively express both global and local scene structures. In our LPD, the upper and lower levels, respectively, represent scene layouts and shape details. We introduce a residual refinement module to progressively complement high-frequency details for signal prediction at each level. To recover the signals at each individual level in the pyramid, an ADFF module is proposed to adaptively fuse multiscale image features for accurate prediction. We conduct comprehensive experiments to evaluate a number of variants of our model on three important DCR tasks, i.e., monocular depth estimation, single-image height estimation, and density map estimation for crowd counting. Experiments demonstrate that our method achieves new state-of-the-art performance in both qualitative and quantitative evaluation on the NYU-D V2 and KITTI for monocular depth estimation, the challenging Urban Semantic 3D (US3D) for satellite height estimation, and four challenging benchmarks for crowd counting. These results demonstrate that the proposed LAPNet is a universal and effective architecture for DCR problems.

Depth Completion Using Laplacian Pyramid-Based Depth Residuals.

Least Square Estimation Network for Depth Completion

RigNet++: Semantic Assisted Repetitive Image Guided Network for Depth Completion

MFF-Net: Towards Efficient Monocular Depth Completion With Multi-Modal Feature Fusion

Deep Unrolled Weighted Graph Laplacian Regularization for Depth Completion

Guided Depth Map Super-Resolution Using Recumbent Y Network

Depth Completion using Plane-Residual Representation

Deep Sparse Depth Completion Using Multi-Scale Residuals and Channel Shuffle

LRRU: Long-short Range Recurrent Updating Networks for Depth Completion

SEHLNet: Separate Estimation of High- and Low-Frequency components for Depth Completion

FCFR-Net: Feature Fusion based Coarse-to-Fine Residual Learning for Depth Completion

A Real-Time Semi-Dense Depth-Guided Depth Completion Network

A Multi-Cue Guidance Network for Depth Completion

Promoting Monocular Depth Estimation by Multi-Scale Residual Laplacian Pyramid Fusion

LapUNet: a novel approach to monocular depth estimation using dynamic laplacian residual U-shape networks

Laplacian Pyramid Neural Network for Dense Continuous-Value Regression for Complex Scenes

Multi-scale Spatial Propagation Network for Depth Completion

Depth Completion from Sparse LiDAR Data with Depth-Normal Constraints

RigNet++: Efficient Repetitive Image Guided Network for Depth Completion

Pyramid Frequency Network with Spatial Attention Residual Refinement Module for Monocular Depth Estimation

DenseLiDAR: A Real-Time Pseudo Dense Depth Guided Depth Completion Network