Index Network

Hao Lu,Yutong Dai,Chunhua Shen,Songcen Xu

DOI: https://doi.org/10.48550/arXiv.1908.09895

2020-04-05

Abstract:We show that existing upsampling operators can be unified using the notion of the index function. This notion is inspired by an observation in the decoding process of deep image matting where indices-guided unpooling can often recover boundary details considerably better than other upsampling operators such as bilinear interpolation. By viewing the indices as a function of the feature map, we introduce the concept of "learning to index", and present a novel index-guided encoder-decoder framework where indices are self-learned adaptively from data and are used to guide the downsampling and upsampling stages, without extra training supervision. At the core of this framework is a new learnable module, termed Index Network (IndexNet), which dynamically generates indices conditioned on the feature map itself. IndexNet can be used as a plug-in applying to almost all off-the-shelf convolutional networks that have coupled downsampling and upsampling stages, giving the networks the ability to dynamically capture variations of local patterns. In particular, we instantiate and investigate five families of IndexNet and demonstrate their effectiveness on four dense prediction tasks, including image denoising, image matting, semantic segmentation, and monocular depth estimation. Code and models have been made available at: <a class="link-external link-https" href="https://tinyurl.com/IndexNetV1" rel="external noopener nofollow">this https URL</a>

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The main objective of this paper is to design a universal upsampling operator to better handle boundary and region prediction issues in different visual tasks. Specifically, the paper proposes a new framework called "IndexNet," whose core idea is to unify existing upsampling methods (such as deconvolution, bilinear interpolation, etc.) into a concept based on index functions. By learning index functions to guide the downsampling and upsampling stages in the encoder-decoder architecture, IndexNet can dynamically and adaptively generate indices from the data without additional supervision information.

The paper points out that traditional upsampling methods perform inconsistently across different tasks, especially in tasks requiring precise boundary information (such as image matting), where methods like bilinear interpolation often fail to recover details. Therefore, the paper proposes a new learning framework that allows the network to adaptively generate indices based on the input feature map, thereby improving the performance of boundary details.

Additionally, the paper explores 5 different variants of IndexNet and demonstrates their effectiveness in 4 dense prediction tasks, including image matting, image denoising, semantic segmentation, and monocular depth estimation. Experimental results show that compared to traditional upsampling methods, IndexNet has significant advantages in these tasks.