Abstract:Learning from a limited amount of data, namely Few-Shot Learning, stands out as a challenging computer vision task. Several works exploit semantics and design complicated semantic fusion mechanisms to compensate for rare representative features within restricted data. However, relying on naive semantics such as class names introduces biases due to their brevity, while acquiring extensive semantics from external knowledge takes a huge time and effort. This limitation severely constrains the potential of semantics in Few-Shot Learning. In this paper, we design an automatic way called Semantic Evolution to generate high-quality semantics. The incorporation of high-quality semantics alleviates the need for complex network structures and learning algorithms used in previous works. Hence, we employ a simple two-layer network termed Semantic Alignment Network to transform semantics and visual features into robust class prototypes with rich discriminative features for few-shot classification. The experimental results show our framework outperforms all previous methods on six benchmarks, demonstrating a simple network with high-quality semantics can beat intricate multi-modal modules on few-shot classification tasks. Code is available at

What problem does this paper attempt to address?

The paper proposes a solution to the few-shot learning (FSL) problem in the field of computer vision. In FSL, the model needs to learn new concepts from limited data. Traditional methods rely on complex network architectures and learning algorithms to extract representative features from images, but this approach is not effective when data is scarce. The paper points out that relying solely on category names (such as nouns) as semantic information may introduce ambiguity, while obtaining rich semantic knowledge from external sources is time-consuming and laborious. To address these issues, the paper introduces a simple framework called "Semantic-Aided Few-Shot Learning (SemFew)". The framework consists of two main parts: Semantic Evolution, which generates high-quality semantic descriptions through an automatic process to complement the limitations of category names; and Semantic Alignment Network (SemAlign), a two-layer network that transforms high-quality semantics and visual features into rich and discriminative class prototypes for more accurate classification. Specifically, Semantic Evolution first converts category names into concise descriptions that match the image content, and then further expands and rewrites these descriptions to include more category-related knowledge. The Semantic Alignment Network does not require a complex semantic understanding module, but rather uses a simple two-layer network to fuse the input semantics and visual features, reconstructing more stable class prototypes. Experimental results demonstrate that SemFew outperforms previous methods on six benchmark datasets, demonstrating that a simple network combined with high-quality semantic information can achieve excellent performance in few-shot classification tasks. The paper also investigates the influence of different types of semantic sources, prototype selection strategies, and classifiers on the results.

Simple Semantic-Aided Few-Shot Learning

Semantic-Based Few-Shot Learning by Interactive Psychometric Testing

Less is More: A Closer Look at Semantic-based Few-Shot Learning

Semantic-Based Few-Shot Classification by Psychometric Learning

Iterative Few-shot Semantic Segmentation from Image Label Text

Imposing Semantic Consistency of Local Descriptors for Few-Shot Learning

Semantic Feature Augmentation in Few-shot Learning.

FewVS: A Vision-Semantics Integration Framework for Few-Shot Image Classification

Improving the Generalised Few-shot Learning by Semantic Information

Semantic Prompt for Few-Shot Image Recognition

Dual Branch Multi-Level Semantic Learning for Few-Shot Segmentation

Semantic-based Selection, Synthesis, and Supervision for Few-shot Learning

Multi-label Few-shot Learning with Semantic Inference (student Abstract)

Semantic Enhanced Few-shot Object Detection

Collect and Select: Semantic Alignment Metric Learning for Few-Shot Learning

Envisioning Class Entity Reasoning by Large Language Models for Few-shot Learning

FILM: How can Few-Shot Image Classification Benefit from Pre-Trained Language Models?

Similarity-Guided and Multi-Layer Fusion Network for Few-shot Semantic Segmentation

Information Symmetry Matters: A Modal-Alternating Propagation Network for Few-Shot Learning

Semantic Mask Reconstruction and Category Semantic Learning for Few-Shot Image Generation

VSA: Adaptive Visual and Semantic Guided Attention on Few-Shot Learning