Abstract:Multi-class supervised learning systems require the knowledge of the entire range of labels they predict. Often when learnt incrementally, they suffer from catastrophic forgetting. To avoid this, generous leeways have to be made to the philosophy of incremental learning that either forces a part of the machine to not learn, or to retrain the machine again with a selection of the historic data. While these hacks work to various degrees, they do not adhere to the spirit of incremental learning. In this article, we redefine incremental learning with stringent conditions that do not allow for any undesirable relaxations and assumptions. We design a strategy involving generative models and the distillation of dark knowledge as a means of hallucinating data along with appropriate targets from past distributions. We call this technique, phantom <a class="link-external link-http" href="http://sampling.We" rel="external noopener nofollow">this http URL</a> show that phantom sampling helps avoid catastrophic forgetting during incremental learning. Using an implementation based on deep neural networks, we demonstrate that phantom sampling dramatically avoids catastrophic forgetting. We apply these strategies to competitive multi-class incremental learning of deep neural networks. Using various benchmark datasets and through our strategy, we demonstrate that strict incremental learning could be achieved. We further put our strategy to test on challenging cases, including cross-domain increments and incrementing on a novel label space. We also propose a trivial extension to unbounded-continual learning and identify potential for future development.

QvQ-IL: Quantity Versus Quality in Incremental Learning

Learning Quality-aware Dynamic Memory for Video Object Segmentation

Incremental Learning for Online Data Using QR Factorization on Convolutional Neural Networks

An Efficient Strategy for Catastrophic Forgetting Reduction in Incremental Learning

Contrastive-based Bi-update Class Incremental Model and Its Application in Fault Diagnosis

Forgetting to Remember: A Scalable Incremental Learning Framework for Cross-Task Blind Image Quality Assessment

Class-Incremental Learning: A Survey

Reducing catastrophic forgetting of incremental learning in the absence of rehearsal memory with task-specific token

Class Incremental Learning with Important and Diverse Memory.

SCMP-IL: an Incremental Learning Method with Super Constraints on Model Parameters

Large Scale Incremental Learning

Maintaining Discrimination and Fairness in Class Incremental Learning

Balanced softmax cross-entropy for incremental learning with and without memory

Quantum Continual Learning Overcoming Catastrophic Forgetting

Multivariate Prototype Representation for Domain-Generalized Incremental Learning

Class-Incremental Learning via Knowledge Amalgamation

Catastrophic forgetting: still a problem for DNNs

Memory Recall: A Simple Neural Network Training Framework Against Catastrophic Forgetting

Reducing Catastrophic Forgetting in Online Class Incremental Learning Using Self-Distillation

A Strategy for an Uncompromising Incremental Learner

ICFD: an Incremental Learning Method Based on Data Feature Distribution.