Abstract:Deep learning-based models have been shown to outperform human beings in many computer vision tasks with massive available labeled training data in learning. However, humans have an amazing ability to easily recognize images of novel categories by browsing only a few examples of these categories. In this case, few-shot learning comes into being to make machines learn from extremely limited labeled examples. One possible reason why human beings can well learn novel concepts quickly and efficiently is that they have sufficient visual and semantic prior knowledge. Toward this end, this work proposes a novel knowledge-guided semantic transfer network (KSTNet) for few-shot image recognition from a supplementary perspective by introducing auxiliary prior knowledge. The proposed network jointly incorporates vision inferring, knowledge transferring, and classifier learning into one unified framework for optimal compatibility. A category-guided visual learning module is developed in which a visual classifier is learned based on the feature extractor along with the cosine similarity and contrastive loss optimization. To fully explore prior knowledge of category correlations, a knowledge transfer network is then developed to propagate knowledge information among all categories to learn the semantic-visual mapping, thus inferring a knowledge-based classifier for novel categories from base categories. Finally, we design an adaptive fusion scheme to infer the desired classifiers by effectively integrating the above knowledge and visual information. Extensive experiments are conducted on two widely used Mini-ImageNet and Tiered-ImageNet benchmarks to validate the effectiveness of KSTNet. Compared with the state of the art, the results show that the proposed method achieves favorable performance with minimal bells and whistles, especially in the case of one-shot learning.

Improving the Generalised Few-shot Learning by Semantic Information

Simple Semantic-Aided Few-Shot Learning

Iterative Few-shot Semantic Segmentation from Image Label Text

Semantic-Based Few-Shot Learning by Interactive Psychometric Testing

Few-shot Image Classification Based on Gradual Machine Learning

Less is More: A Closer Look at Semantic-based Few-Shot Learning

Knowledge Graph Enhanced Multimodal Learning for Few-shot Visual Recognition

Knowledge-Guided Semantic Transfer Network for Few-Shot Image Recognition

FILM: How can Few-Shot Image Classification Benefit from Pre-Trained Language Models?

Semantic Mask Reconstruction and Category Semantic Learning for Few-Shot Image Generation

VSA: Adaptive Visual and Semantic Guided Attention on Few-Shot Learning

Semantic-Based Few-Shot Classification by Psychometric Learning

Enhancing Generalized Few-Shot Semantic Segmentation via Effective Knowledge Transfer

Better Integrating Vision and Semantics for Improving Few-shot Classification

SgVA-CLIP: Semantic-Guided Visual Adapting of Vision-Language Models for Few-Shot Image Classification

FewVS: A Vision-Semantics Integration Framework for Few-Shot Image Classification

Learn from Concepts: Towards the Purified Memory for Few-shot Learning.

Harnessing Multi-Semantic Hypergraph for Few-Shot Learning.

Multi-label Few-shot Learning with Semantic Inference (student Abstract)

Textual Enhanced Adaptive Meta-Fusion for Few-Shot Visual Recognition

Attention-Based Multi-Context Guiding for Few-Shot Semantic Segmentation