Abstract: Superpixels have been widely used in computer vision tasks due to their representational and computational efficiency. Meanwhile, deep learning and end-to-end framework have made great progress in various fields including computer vision. However, existing superpixel algorithms cannot be integrated into subsequent tasks in an end-to-end way. Traditional algorithms and deep learning-based algorithms are two main streams in superpixel segmentation. The former is non-differentiable and the latter needs a non-differentiable post-processing step to enforce connectivity, which constraints the integration of superpixels and downstream tasks. In this paper, we propose a deep learning-based superpixel segmentation algorithm SIN which can be integrated with downstream tasks in an end-to-end way. Owing to some downstream tasks such as visual tracking require real-time speed, the speed of generating superpixels is also important. To remove the post-processing step, our algorithm enforces spatial connectivity from the start. Superpixels are initialized by sampled pixels and other pixels are assigned to superpixels through multiple updating steps. Each step consists of a horizontal and a vertical interpolation, which is the key to enforcing spatial connectivity. Multi-layer outputs of a fully convolutional network are utilized to predict association scores for interpolations. Experimental results show that our approach runs at about 80fps and performs favorably against state-of-the-art methods. Furthermore, we design a simple but effective loss function which reduces much training time. The improvements of superpixel-based tasks demonstrate the effectiveness of our algorithm. We hope SIN will be integrated into downstream tasks in an end-to-end way and benefit the superpixel-based community. Code is available at: \href{https://github.com/yuanqqq/SIN}{https://github.com/yuanqqq/SIN}.

AINet+: Advancing Superpixel Segmentation via Cascaded Association Implantation

AINet: Association Implantation for Superpixel Segmentation

Deep Superpixel Convolutional Network for Image Recognition

ISDNet: Integrating Shallow and Deep Networks for Efficient Ultra-high Resolution Segmentation

Improving Semantic Image Segmentation with a Probabilistic Superpixel-Based Dense Conditional Random Field.

Rethinking Superpixel Segmentation from Biologically Inspired Mechanisms

SIN - Superpixel Interpolation Network.

Superpixel Segmentation with Squeeze-and-excitation Networks

Superpixel Segmentation with Attention Convolution Neural Network

Superpixel Guided Network for Weakly Supervised Semantic Segmentation

An Object-Aware Network Embedding Deep Superpixel for Semantic Segmentation of Remote Sensing Images

Superpixel Segmentation Based on Square-Wise Asymmetric Partition and Structural Approximation.

Adaptive High-Precision Superpixel Segmentation

MSSINet: Real-Time Segmentation Based on Multi-Scale Strip Integration

Image Semantic Segmentation Algorithm Based on Self-learning Super-Pixel Feature Extraction.

Cascaded Multiscale Structure with Self-Smoothing Atrous Convolution for Semantic Segmentation

Superdense-scale Network for Semantic Segmentation

ConnNet: A Long-Range Relation-Aware Pixel-Connectivity Network for Salient Segmentation.

CONIC: Contour Optimized Non-Iterative Clustering Superpixel Segmentation

Automatic segmentation algorithm for high-spatial-resolution remote sensing images based on self-learning super-pixel convolutional network

DEL: Deep Embedding Learning for Efficient Image Segmentation.