Abstract:Data hallucination generates additional training examples for novel classes to alleviate the data scarcity problem in few-shot learning (FSL). Existing hallucination-based FSL methods normally train a general embedding model first by applying information extracted from base classes that have abundant data. In those methods, hallucinators are then built upon the trained embedding model to generate data for novel classes. However, these hallucination methods usually rely on general-purpose embeddings, limiting their ability to generate task-oriented samples for novel classes. Recent studies have shown that task-specific embedding models, which are adapted to novel tasks, can achieve better classification performance. To improve the performance of example hallucination for tasks, a task-oriented embedding model is used in the proposed method to perform task-oriented generation. After the initialization, the hallucinator is finetuned by applying a task-oriented embedding model with the guidance of a teacher-student mechanism. The proposed task-oriented hallucination method contains two steps. An initial embedding network and an initial hallucinator are trained with a base dataset in the first step. The second step contains a pseudo-labelling process where the base dataset is pseudo-labelled using support data of the few-shot task and a task-oriented fine-tuning process where the embedding network and hallucinator are adjusted simultaneously. Both the embedding network and the hallucinator are updated with the support set and the pseudo-labelled base dataset using knowledge distillation. The experiments are conducted on four popular few-shot datasets. The results demonstrate that the proposed approach outperforms state-of-the-art methods with 0.8% to 4.08% increases in classification accuracy for 5-way 5-shot tasks. It also achieves comparable accuracy to state-of-the-art methods for 5-way 1-shot tasks.

Learning to Memorize Feature Hallucination for One-Shot Image Generation

TcGAN: Semantic-Aware and Structure-Preserved GANs with Individual Vision Transformer for Fast Arbitrary One-Shot Image Generation

Task-oriented Feature Hallucination for Few-Shot Image Classification.

MemoryGAN: GAN Generator As Heterogeneous Memory for Compositional Image Synthesis

Memory Replay GANs: Learning to Generate Images from New Categories Without Forgetting

Memory Matching Networks for One-Shot Image Recognition

Multi-Level Semantic Feature Augmentation for One-Shot Learning

One-shot Learning with Memory-Augmented Neural Networks

Dual-View Data Hallucination with Semantic Relation Guidance for Few-Shot Image Recognition

Discriminative learning of imaginary data for few-shot classification

Learning to Generate with Memory

Spatial-Aware GAN for Instance-Guided Cross-Spectral Face Hallucination

Saliency-guided meta-hallucinator for few-shot learning

Hallucination Improves the Performance of Unsupervised Visual Representation Learning

VOCABULARY-INFORMED VISUAL FEATURE AUGMENTATION FOR ONE-SHOT LEARNING

Heterogenous Memory Augmented Neural Networks

Adaptive Forgetting, Drafting and Comprehensive Guiding: Text-to-Image Synthesis with Hierarchical Generative Adversarial Networks

Few-Shot Semantic Segmentation with Cyclic Memory Network

Semantics-Guided Intra-Category Knowledge Transfer for Generalized Zero-Shot Learning

Generative Cross-Modal Retrieval: Memorizing Images in Multimodal Language Models for Retrieval and Beyond

Memory-guided Network with Uncertainty-based Feature Augmentation for Few-shot Semantic Segmentation