Conservative Generator, Progressive Discriminator: Coordination of Adversaries in Few-shot Incremental Image Synthesis

Chaerin Kong,Nojun Kwak

2024-11-16

Abstract:The capacity to learn incrementally from an online stream of data is an envied trait of human learners, as deep neural networks typically suffer from catastrophic forgetting and stability-plasticity dilemma. Several works have previously explored incremental few-shot learning, a task with greater challenges due to data constraint, mostly in classification setting with mild success. In this work, we study the underrepresented task of generative incremental few-shot learning. To effectively handle the inherent challenges of incremental learning and few-shot learning, we propose a novel framework named ConPro that leverages the two-player nature of GANs. Specifically, we design a conservative generator that preserves past knowledge in parameter and compute efficient manner, and a progressive discriminator that learns to reason semantic distances between past and present task samples, minimizing overfitting with few data points and pursuing good forward transfer. We present experiments to validate the effectiveness of ConPro.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper aims to solve the problem of Generative Incremental Few - shot Learning (GIL). Specifically, it attempts to combine the challenges in two fields: Incremental Learning (IL) and Few - shot Learning (FSL).

1. **Catastrophic Forgetting in Incremental Learning**: In incremental learning, the model needs to learn new tasks without forgetting the knowledge of past tasks. However, traditional incremental learning methods often lead to the model forgetting the knowledge of old tasks when learning new tasks, and this phenomenon is called "catastrophic forgetting".

2. **Overfitting in Few - shot Learning**: In few - shot learning, because the amount of available data is very limited, the model is prone to overfitting to the training data, resulting in poor generalization ability.

To address these challenges, the author proposes a new framework - **ConPro** (Conservative Generator, Progressive Discriminator), which takes advantage of the two - player characteristics of Generative Adversarial Networks (GANs):

- **Conservative Generator**: A generator with high parameter and computational efficiency is designed. It retains past knowledge through task - specific module expansion and serves as a memory replay buffer.

- **Progressive Discriminator**: A discriminator that can learn the semantic distance between current - task and past - task samples is designed to reduce overfitting and promote good forward transfer.

Through the synergy of these two components, the ConPro framework can perform excellently in solving the dual challenges of incremental learning and few - shot learning.