Abstract:Deep learning-based methods have recently advanced image generation by exploiting the valuable information within immense training data, but they struggle to synthesize new images for rare categories that are scarce enough to cover some visual concepts. Image generation based on few-shot learning is developed to capture the generalizable generative feature from limited data to produce new images for unseen categories. Existing few-shot methods focus on the high-level semantic difference between conditional images and fuse a generative feature based on the semantic metric. However, it ignores the impact of semantic information underlying different level feature subspaces throughout the generation process, leading to the degradation of visual quality when the diversity of synthetic images improves. In this work, we propose a novel Adaptive Multi-scale Modulation Generative Adversarial Network (AMMGAN) for few-shot image generation, leveraging the U-Net with skip connections to exploit multi-scale semantic metric information. Specifically, an adaptive self-metric fusion module is introduced at the junction between the encoder and decoder of generator, which measures the pixel-wise semantic information of conditional images as a self-metric attention map in each level of deep features to fuse a general feature of interest and adaptively adjust the mean and variance of the fused feature based on the high-level semantic feature of decoder. Meanwhile, the part network of discriminator is developed to co-learn with the generator to formulate a reference-metric modulation code embedding channel-wise metric information, which is integrated into the decoder of generator by learnable residual transformation to refine the unsampled fused feature from the channel dimension. Extensive experiments on several benchmark datasets demonstrate the effectiveness of AMMGAN on few-shot image generation and downstream visual classification tasks.

F2GAN: Fusing-and-Filling GAN for Few-shot Image Generation.

Exact Fusion Via Feature Distribution Matching for Few-shot Image Generation

Matchinggan: Matching-Based Few-Shot Image Generation

CGMGM: A Cross-Gaussian Mixture Generative Model for Few-Shot Semantic Segmentation

FedGAN: Federated GAN for Few-shot Image Generation

EqGAN: Feature Equalization Fusion for Few-shot Image Generation

Few-Shot Adaptation of Generative Adversarial Networks

AMMGAN: Adaptive Multi-Scale Modulation Generative Adversarial Network for Few-Shot Image Generation

LoFGAN: Fusing Local Representations for Few-shot Image Generation

Semantic-Aware Generator and Low-level Feature Augmentation for Few-shot Image Generation

DFSGAN: Introducing editable and representative attributes for few-shot image generation

High-Quality and Diverse Few-Shot Image Generation via Masked Discrimination

DeltaGAN: Towards Diverse Few-Shot Image Generation with Sample-Specific Delta.

Local-Fusion Diffusion Model for Enhancing Few-Shot Image Generation.

A Feature Generator for Few-Shot Learning

D3T-GAN: Data-Dependent Domain Transfer GANs for Few-shot Image Generation

Fast Adaptive Meta-Learning for Few-Shot Image Generation

Few-shot Image Generation via Information Transfer from the Built Geodesic Surface

LFS-GAN: Lifelong Few-Shot Image Generation

ProtoGAN: Towards High Diversity and Fidelity Image Synthesis under Limited Data

WaveGAN: Frequency-Aware GAN for High-Fidelity Few-Shot Image Generation