Abstract:One-shot semantic image segmentation aims to segment the object regions for the novel class with only one annotated image. Recent works adopt the episodic training strategy to mimic the expected situation at testing time. However, these existing approaches simulate the test conditions too strictly during the training process, and thus cannot make full use of the given label information. Besides, these approaches mainly focus on the foreground-background target class segmentation setting. They only utilize binary mask labels for training. In this paper, we propose to leverage the multi-class label information during the episodic training. It will encourage the network to generate more semantically meaningful features for each category. After integrating the target class cues into the query features, we then propose a pyramid feature fusion module to mine the fused features for the final classifier. Furthermore, to take more advantage of the support image-mask pair, we propose a self-prototype guidance branch to support image segmentation. It can constrain the network for generating more compact features and a robust prototype for each semantic class. For inference, we propose a fused prototype guidance branch for the segmentation of the query image. Specifically, we leverage the prediction of the query image to extract the pseudo-prototype and combine it with the initial prototype. Then we utilize the fused prototype to guide the final segmentation of the query image. Extensive experiments demonstrate the superiority of our proposed approach. The source codes and models have been made available at https://github.com/NUST-Machine-Intelligence-Laboratory/SMCP.

Multi-Level Semantic Feature Augmentation for One-Shot Learning

Semantic Feature Augmentation in Few-shot Learning.

VOCABULARY-INFORMED VISUAL FEATURE AUGMENTATION FOR ONE-SHOT LEARNING

Multi-modal Generative Adversarial Network for Zero-Shot Learning

Multi-Attention Network For One Shot Learning

Multi-level Fusion of Multi-modal Semantic Embeddings for Zero Shot Learning

Semantically Meaningful Class Prototype Learning for One-Shot Image Semantic Segmentation

Dual Branch Multi-Level Semantic Learning for Few-Shot Segmentation

Dual Feature Augmentation Network for Generalized Zero-shot Learning

Semantically Meaningful Class Prototype Learning for One-Shot Image Segmentation

Adaptive multi-scale semantic fusion network for zero-shot learning

Simple Semantic-Aided Few-Shot Learning

Multi-Semantic Hypergraph Neural Network for Effective Few-Shot Learning

TPSN: Transformer-based Multi-Prototype Search Network for Few-Shot Semantic Segmentation

Two-Branch Attention Network via Efficient Semantic Coupling for One-Shot Learning

Harnessing Multi-Semantic Hypergraph for Few-Shot Learning.

Dual-View Data Hallucination with Semantic Relation Guidance for Few-Shot Image Recognition

Multiscale Attention-Based Prototypical Network for Few-Shot Semantic Segmentation.

One-shot Learning with Memory-Augmented Neural Networks

Attribute Prototype Network for Any-Shot Learning