Abstract:We propose Sym-Net, a novel framework for Few-Shot Segmentation (FSS) that addresses the critical issue of intra-class variation by jointly learning both query and support prototypes in a symmetrical manner. Unlike previous methods that generate query prototypes solely by matching query features to support prototypes, which is a form of bias learning towards the few-shot support samples, Sym-Net leverages a balanced symmetrical learning approach for both query and support prototypes, ensuring that the learning process does not favor one set (support or query) over the other. One of main modules of Sym-Net is the visual-text alignment-based prototype aggregation module, which is not just query-guided prototype refinement, it is a jointly learning from both support and query samples, which makes the model beneficial for handling intra-class discrepancies and allows it to generalize better to new, unseen classes. Specifically, a parameter-free prior mask generation module is designed to accurately localize both local and global regions of the query object by using sliding windows of different sizes and a self-activation kernel to suppress incorrect background matches. Additionally, to address the information loss caused by spatial pooling during prototype learning, a top-down hyper-correlation module is integrated to capture multi-scale spatial relationships between support and query images. This approach is further jointly optimized by implementing a co-optimized hard triplet mining strategy. Experimental results show that the proposed Sym-Net outperforms state-of-the-art models, which demonstrates that jointly learning support-query prototypes in a symmetrical manner for FSS offers a promising direction to enhance segmentation performance with limited annotated data.

Learning Equivariant Segmentation with Instance-Unique Querying

QueryInst: Parallelly Supervised Mask Query for Instance Segmentation

SOLQ: Segmenting Objects by Learning Queries

DropQueries: A Simple Way to Discover Comprehensive Segment Representations

A Unified Query-based Paradigm for Camouflaged Instance Segmentation

Query Learning of Both Thing and Stuff for Panoptic Segmentation.

Adaptive Query Selection for Camouflaged Instance Segmentation

Learning Task-Aligned Mask Query for Instance Segmentation

3D-QueryIS: A Query-based Framework for 3D Instance Segmentation

Enhancing Query Formulation for Universal Image Segmentation

Mask Matching Transformer for Few-Shot Segmentation

Instances As Queries

Learning Co-segmentation by Segment Swapping for Retrieval and Discovery

Continual Learning for Image Segmentation with Dynamic Query

Learning Segmentation Masks with the Independence Prior

SGIFormer: Semantic-guided and Geometric-enhanced Interleaving Transformer for 3D Instance Segmentation

DQFormer: Towards Unified LiDAR Panoptic Segmentation with Decoupled Queries

MDQE: Mining Discriminative Query Embeddings to Segment Occluded Instances on Challenging Videos

Symmetrical Joint Learning Support-query Prototypes for Few-shot Segmentation

A Two-Pipeline Instance Segmentation Network via Boundary Enhancement for Scene Understanding

CQformer: Learning Dynamics Across Slices in Medical Image Segmentation