Abstract:Semantic segmentation is a fundamental problem in multimedia which requires delicate per-pixel predictions of object categories. Recently, many researchers strive to refine the pixel-wise feature with spatial -contextual information. However, many of them still neglect the invisible hand of cross- channel information which provides inherent semantics to facilitate the segmentation performance. On the one hand, in the feature extraction stage, enhancing informative channels and suppressing trivial ones contribute to the acquisition of valuable semantic features, and thus improving the segmentation accuracy. On the other hand, in the prediction stage, we can predict the complete objects more clearly by finding the connections and complements between different channels, which can also contribute to the pixel prediction. And based on this idea, we propose a novel Channel-Adaptive Network for semantic segmentation, which is capable of enhancing the features from the perspective of channels in both feature extraction stage and prediction stage. Specifically, we propose two modules: (i) the Comprehensive Information Channel Attention (CiCA) module that addresses the shortcomings of existing channel attention by learning both low and high frequency components within each channel for emphasizing the informative channels; (ii) the Inter-Channel Relationship Reasoning (iCRR) module which is applied on the top of the feature extractor to adaptively enhance the interdependent channels by mining the complementary associations between them. Besides, our Channel-Adaptive Network is highly flexible, with a plug-and-play design. Extensive experiments have demonstrated that our method achieves the state-of-the-art segmentation performance on three challenging datasets, including Cityscapes (82.1%), ADE20K (46.51%) and PASCAL Context (55.0%).

Frozen is better than learning: A new design of prototype-based classifier for semantic segmentation

LEARNING SHAPE PRIORS BY PAIRWISE COMPARISON FOR ROBUST SEMANTIC SEGMENTATION

Prototype-based Semantic Segmentation

Rethinking Semantic Segmentation: A Prototype View

FrozenSeg: Harmonizing Frozen Foundation Models for Open-Vocabulary Segmentation

Semantic and Spatial Adaptive Pixel-level Classifier for Semantic Segmentation

Multi-Modal Prototypes for Open-World Semantic Segmentation

Learning Context-aware Classifier for Semantic Segmentation

SSA-Seg: Semantic and Spatial Adaptive Pixel-level Classifier for Semantic Segmentation

Frozen CLIP: A Strong Backbone for Weakly Supervised Semantic Segmentation

RSProtoSeg: High Spatial Resolution Remote Sensing Images Segmentation Based on Non-Learnable Prototypes

Better Image Segmentation by Exploiting Dense Semantic Predictions

Learning and Memorizing Representative Prototypes for 3D Point Cloud Semantic and Instance Segmentation

Subspace Prototype Guidance for Mitigating Class Imbalance in Point Cloud Semantic Segmentation

Background Adaptation with Residual Modeling for Exemplar-Free Class-Incremental Semantic Segmentation

RBC: Rectifying the Biased Context in Continual Semantic Segmentation

Dual Prototypical Contrastive Learning for Few-shot Semantic Segmentation

Bidirectional Self-Training with Multiple Anisotropic Prototypes for Domain Adaptive Semantic Segmentation

Learning Cross-Channel Representations for Semantic Segmentation

Prototype Adaption and Projection for Few- and Zero-Shot 3D Point Cloud Semantic Segmentation