Depth Image Super-resolution via Two-Branch Network

Jiaxin Guo,Rong Xiong,Yongsheng Ou,Lin Wang,Chao Liu
DOI: https://doi.org/10.1007/978-981-16-9247-5_15
2022-01-01
Abstract:With the continuous development of sensors, a number of inexpensive and effective depth cameras have shown up, which have greatly contributed to the development of autonomous driving and 3D reconstruction technologies. However, the depth images captured by low-cost depth cameras are low-resolution, which is difficult to meet the needs of practical applications. We propose a two-branch network to achieve depth map super-resolution with high-resolution guidance image, which can be viewed as a prior to guide the low-resolution depth map to restore the missing high-frequency details of structures. To emphasize the guidance role of high-resolution images, we use spatially-variant kernels based on the guidance feature map to replace the original convolution kernels. In addition, in order to extract the feature maps of the depth images more effectively, we add the channel attention mechanism between convolution layers. Our network is trained end-to-end, supporting various sizes of input images because the network backbone uses full convolution and no fully connected layers. The proposal model is trained only on a certain dataset under three super-resolution factors and utilized directly on other datasets without fine-tuning. We show the effectiveness of our model by comparing it with state-of-art methods.
What problem does this paper attempt to address?