Abstract:Generative Adversarial Networks (GANs) have shown remarkable performance in image synthesis tasks, but typically require a large number of training samples to achieve high-quality synthesis. This paper proposes a simple and effective method, Few-Shot GAN (FSGAN), for adapting GANs in few-shot settings (less than 100 images). FSGAN repurposes component analysis techniques and learns to adapt the singular values of the pre-trained weights while freezing the corresponding singular vectors. This provides a highly expressive parameter space for adaptation while constraining changes to the pretrained weights. We validate our method in a challenging few-shot setting of 5-100 images in the target domain. We show that our method has significant visual quality gains compared with existing GAN adaptation methods. We report qualitative and quantitative results showing the effectiveness of our method. We additionally highlight a problem for few-shot synthesis in the standard quantitative metric used by data-efficient image synthesis works. Code and additional results are available at <a class="link-external link-http" href="http://e-271.github.io/few-shot-gan" rel="external noopener nofollow">this http URL</a>.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: in the case of a small number of samples (less than 100 images), how to effectively adapt Generative Adversarial Networks (GANs) to generate high - quality images. Traditional GANs usually require a large number of training samples to achieve high - quality image synthesis, but in practical applications, obtaining a large amount of high - quality image data may be both expensive and infeasible. Therefore, this research proposes a simple and effective method - Few - Shot GAN (FSGAN) for adjusting pre - trained GAN models in the case of a small number of samples.
### Specific Problem Description
1. **GAN Adaptability Problem in the Case of Small Samples**:
- Traditional GAN models usually require a large amount of image data during training to ensure the quality of the generated images. However, in many practical application scenarios, especially in specific fields or for specific tasks, only a small number of training samples may be available.
- GAN training in the case of small samples is prone to problems such as over - fitting, lack of diversity, or poor image quality.
2. **Limitations of Existing Methods**:
- Existing GAN adaptation methods such as fine - tuning the entire network weights (Fine - tuning), only adjusting batch normalization statistics (Batch Norm Statistics), etc., have poor performance in the case of small samples, especially when the source domain and the target domain are quite different.
- These methods either result in blurred generated images with insufficient details or are difficult to capture the feature changes of the target domain.
### Solution
The paper proposes a method based on Singular Value Decomposition (SVD) to adapt to the new target domain by adjusting the singular values in the pre - trained GAN model. The specific steps are as follows:
- **Singular Value Decomposition (SVD)**: Perform SVD decomposition on the weights of the convolutional layers and fully - connected layers of the pre - trained GAN to obtain the left singular vector \( U \), the right singular vector \( V \) and the singular value matrix \( \Sigma \).
- **Adapting Singular Values**: Freeze the left singular vector \( U \) and the right singular vector \( V \), and only optimize the singular value matrix \( \Sigma \), so as to achieve semantically meaningful changes while maintaining the natural structure.
- **Training and Inference**: In the forward propagation process, use the optimized singular values to reconstruct the weights and apply them to the operations of the convolutional or fully - connected layers.
Through this method, FSGAN can generate high - quality and diverse images in the case of a small number of samples, while avoiding the problems of over - fitting and image quality degradation.
### Experimental Verification
The paper verifies the effectiveness of FSGAN through multiple experiments, including near - domain adaptation (such as FFHQ to single - ID CelebA) and far - domain adaptation (such as LSUN Churches to Van Gogh paintings). The experimental results show that FSGAN can produce high - quality adaptation effects in various n - shot settings and is superior to existing GAN adaptation methods.
### Summary
This paper proposes a novel and effective small - sample GAN adaptation method FSGAN. By adjusting the singular values in the pre - trained GAN model, it achieves the goal of generating high - quality images in the case of a small number of samples. This method provides a new idea for solving small - sample generation tasks and shows its wide applicability in different domain adaptations.