MSF-Net: Multi-Scale Feedback Reconstruction for Guided Depth Map Super-Resolution
Jin Wang,Chenyang Li,Yunhui Shi,Dan Wang,Mu-En Wu,Nam Ling,Baocai Yin
DOI: https://doi.org/10.1109/tcsvt.2023.3288339
IF: 5.859
2023-01-01
IEEE Transactions on Circuits and Systems for Video Technology
Abstract:Guided depth map super-resolution (GDSR) is one of the mainstream methods in depth map super-resolution, as high-resolution color images can guide the reconstruction of the depth maps and are often easy to obtain. However, how to make full use of extracted guidance information of the color image to improve the depth map reconstruction remains a challenging problem. In this paper, we first design a multi-scale feedback module (MF) that extracts multi-scale features and alleviates the information loss in network propagation. We further propose a novel multi-scale feedback network (MSF-Net) for guided depth map super-resolution, which can better extract and refine the features by sequentially joining MF blocks. Specifically, our MF block uses parallel sampling layers and feedback links between multiple time steps to better learn information at different scales. Moreover, an inter-scale attention module (IA) is proposed to adaptively select and fuse important features at different scales. Meanwhile, depth features and corresponding color features are interacted using cross-domain attention conciliation module (CAC) after each MF block. We evaluate the performance of our proposed method on both synthetic and real captured datasets. Extensive experimental results validate that the proposed method achieves state-of-the-art performance in both objective and subjective quality.
engineering, electrical & electronic