Abstract:Research into Few-shot Semantic Segmentation (FSS) has attracted great attention, with the goal to segment target objects in a query image given only a few annotated support images of the target class. A key to this challenging task is to fully utilize the information in the support images by exploiting fine-grained correlations between the query and support images. However, most existing approaches either compressed the support information into a few class-wise prototypes, or used partial support information (e.g., only foreground) at the pixel level, causing non-negligible information loss. In this paper, we propose Dense pixel-wise Cross-query-and-support Attention weighted Mask Aggregation (DCAMA), where both foreground and background support information are fully exploited via multi-level pixel-wise correlations between paired query and support features. Implemented with the scaled dot-product attention in the Transformer architecture, DCAMA treats every query pixel as a token, computes its similarities with all support pixels, and predicts its segmentation label as an additive aggregation of all the support pixels’ labels—weighted by the similarities. Based on the unique formulation of DCAMA, we further propose efficient and effective one-pass inference for n-shot segmentation, where pixels of all support images are collected for the mask aggregation at once. Experiments show that our DCAMA significantly advances the state of the art on standard FSS benchmarks of PASCAL-5 $$^i$$ , COCO-20 $$^i$$ , and FSS-1000, e.g., with 3.1%, 9.7%, and 3.6% absolute improvements in 1-shot mIoU over previous best records. Ablative studies also verify the design DCAMA.

A Self-Distillation Embedded Supervised Affinity Attention Model for Few-Shot Segmentation

Dense Cross-Query-and-Support Attention Weighted Mask Aggregation for Few-Shot Segmentation

Boosting Few-Shot Segmentation via Instance-Aware Data Augmentation and Local Consensus Guided Cross Attention

Learning self-target knowledge for few-shot segmentation

Self-Calibrated Cross Attention Network for Few-Shot Segmentation

Few-shot object segmentation with a new feature aggregation module

Few-Shot Semantic Segmentation with Democratic Attention Networks

Self-Support Few-Shot Semantic Segmentation

APANet: Adaptive Prototypes Alignment Network for Few-Shot Semantic Segmentation

Distilling base-and-meta network with contrastive learning for few-shot semantic segmentation

Iterative Few-shot Semantic Segmentation from Image Label Text

Self-Support Matching Networks with Multiscale Attention for Few-shot Semantic Segmentation

Self-Enhanced Mixed Attention Network for Three-Modal Images Few-Shot Semantic Segmentation

Multi-scale Self-similarity Network for Few-Shot Segmentation

A Joint Framework Towards Class-aware and Class-agnostic Alignment for Few-shot Segmentation

SCNet: Enhancing Few-Shot Semantic Segmentation by Self-Contrastive Background Prototypes

Unleashing the Potential of the Diffusion Model in Few-shot Semantic Segmentation

Cycle association prototype network for few-shot semantic segmentation

Rethinking and Improving Few-Shot Segmentation from a Contour-Aware Perspective

A Mutually Supervised Graph Attention Network for Few-Shot Segmentation: The Perspective of Fully Utilizing Limited Samples